Phase-Technologies/HyperSafe-Deep-ZSC-Ncert

HyperSafe Deep Zero-Shot Classifier (ZSC) - Definitive Technical Whitepaper

1. Formal Performance Benchmark

• Evaluation Set: 100 Manually Crafted Cross-Domain Queries
• Global Accuracy Score: 40.00%
• Metric: Cosine Similarity Top-1 Accuracy
• Inference Latency: ~12ms per query (Tesla T4)

Domain	Status	Observations
History	High	Strong alignment on temporal and era-based keywords.
Sports	High	Excellent categorization of game-related terminology.
Science	Low	High variance in nomenclature; requires further fine-tuning.
Math	Medium	Moderate recognition of symbolic descriptions.

2. Structural Decomposition & Layer Analysis

2.1 Transformer Block Topology

The model implements a 'DeepSafe' variant of the Transformer Encoder (Vaswani et al.). It consists of 12 stacked layers, utilizing Pre-Layer Normalization to prevent gradient vanishing in the 256-dimensional embedding space.

2.2 Latent Space Geometry

The output of the pooler is projected onto a 256-D hypersphere. Similarity is calculated via: $$\text{score}=\frac{E_{text}\cdot E_{label}}{\mathrm{\parallel }{E}_{text}\mathrm{\parallel }\mathrm{\parallel }{E}_{label}\mathrm{\parallel }}\backslash text\{score\}\; =\; \backslash frac\{E\_\{text\}\; \backslash cdot\; E\_\{label\}\}\{\backslash Vert\; E\_\{text\}\; \backslash Vert\; \backslash Vert\; E\_\{label\}\; \backslash Vert\}$$

2.3 Weight Distribution Audit (Real Data)

Below is the audit of the current state of the model parameters:

• token_embed.weight: Mean=0.000414, Std=1.000097, Shape=[50257, 256]
• encoder.layers.0.self_attn.in_proj_weight: Mean=-0.000031, Std=0.044178, Shape=[768, 256]
• encoder.layers.0.self_attn.out_proj.weight: Mean=-0.000229, Std=0.036035, Shape=[256, 256]
• encoder.layers.0.linear1.weight: Mean=-0.000042, Std=0.036081, Shape=[1024, 256]
• encoder.layers.0.linear2.weight: Mean=0.000054, Std=0.018051, Shape=[256, 1024]
• encoder.layers.0.norm1.weight: Mean=0.999191, Std=0.000611, Shape=[256]
• encoder.layers.0.norm2.weight: Mean=1.001201, Std=0.000516, Shape=[256]
• encoder.layers.1.self_attn.in_proj_weight: Mean=-0.000031, Std=0.044177, Shape=[768, 256]
• encoder.layers.1.self_attn.out_proj.weight: Mean=-0.000230, Std=0.036035, Shape=[256, 256]
• encoder.layers.1.linear1.weight: Mean=-0.000039, Std=0.036064, Shape=[1024, 256]
• encoder.layers.1.linear2.weight: Mean=0.000049, Std=0.018045, Shape=[256, 1024]
• encoder.layers.1.norm1.weight: Mean=0.999228, Std=0.000830, Shape=[256]
• encoder.layers.1.norm2.weight: Mean=1.000884, Std=0.000621, Shape=[256]
• encoder.layers.2.self_attn.in_proj_weight: Mean=-0.000032, Std=0.044179, Shape=[768, 256]
• encoder.layers.2.self_attn.out_proj.weight: Mean=-0.000229, Std=0.036039, Shape=[256, 256]
• encoder.layers.2.linear1.weight: Mean=-0.000039, Std=0.036051, Shape=[1024, 256]
• encoder.layers.2.linear2.weight: Mean=0.000045, Std=0.018042, Shape=[256, 1024]
• encoder.layers.2.norm1.weight: Mean=0.999338, Std=0.000969, Shape=[256]
• encoder.layers.2.norm2.weight: Mean=1.000600, Std=0.000859, Shape=[256]
• encoder.layers.3.self_attn.in_proj_weight: Mean=-0.000032, Std=0.044179, Shape=[768, 256]
• encoder.layers.3.self_attn.out_proj.weight: Mean=-0.000230, Std=0.036046, Shape=[256, 256]
• encoder.layers.3.linear1.weight: Mean=-0.000040, Std=0.036045, Shape=[1024, 256]
• encoder.layers.3.linear2.weight: Mean=0.000042, Std=0.018041, Shape=[256, 1024]
• encoder.layers.3.norm1.weight: Mean=0.999406, Std=0.001058, Shape=[256]
• encoder.layers.3.norm2.weight: Mean=1.000430, Std=0.001025, Shape=[256]
• encoder.layers.4.self_attn.in_proj_weight: Mean=-0.000031, Std=0.044182, Shape=[768, 256]
• encoder.layers.4.self_attn.out_proj.weight: Mean=-0.000231, Std=0.036053, Shape=[256, 256]
• encoder.layers.4.linear1.weight: Mean=-0.000040, Std=0.036043, Shape=[1024, 256]
• encoder.layers.4.linear2.weight: Mean=0.000040, Std=0.018042, Shape=[256, 1024]
• encoder.layers.4.norm1.weight: Mean=0.999490, Std=0.001058, Shape=[256]
• encoder.layers.4.norm2.weight: Mean=1.000360, Std=0.001154, Shape=[256]
• encoder.layers.5.self_attn.in_proj_weight: Mean=-0.000031, Std=0.044183, Shape=[768, 256]
• encoder.layers.5.self_attn.out_proj.weight: Mean=-0.000232, Std=0.036060, Shape=[256, 256]
• encoder.layers.5.linear1.weight: Mean=-0.000039, Std=0.036042, Shape=[1024, 256]
• encoder.layers.5.linear2.weight: Mean=0.000038, Std=0.018043, Shape=[256, 1024]
• encoder.layers.5.norm1.weight: Mean=0.999542, Std=0.001044, Shape=[256]
• encoder.layers.5.norm2.weight: Mean=1.000320, Std=0.001187, Shape=[256]
• encoder.layers.6.self_attn.in_proj_weight: Mean=-0.000031, Std=0.044185, Shape=[768, 256]
• encoder.layers.6.self_attn.out_proj.weight: Mean=-0.000232, Std=0.036066, Shape=[256, 256]
• encoder.layers.6.linear1.weight: Mean=-0.000038, Std=0.036043, Shape=[1024, 256]

3. Fast Markov Pre-Scoring Mechanics

Before the deep encoder processes the text, a 2nd-order Markov chain estimates sequence probability.

• Order: Trigram ($n=2$)
• Vocabulary Depth: 50,257 (BPE-aligned)
• Smoothing: Laplace (+0.1) applied to transition counts to handle Out-of-Vocabulary (OOV) tokens.

4. Formal Usage and Safety Protocol

This model is intended for academic research in Zero-Shot Learning.

Checkpoint Loading

model = DeepSafeEncoder()
model.load_state_dict(torch.load('hyper_zsc_model.pt'))