Search mathematical notation by meaning Symbol familiesVertical bar notationScalar multiplication notationFraction notationSquare root notationEta notationEquals notationDefined as notationEquivalent notationApproximately equal notationProportional to notationNot equal notationLess than notationLess than or equal notationMuch less than notationMaps to notationRight arrow notationSubscript notationSuperscript notationPrime mark notationTranspose notationHat notationOverbar notationDot accent notationBold symbol notationCalligraphic symbol notationReal numbers notationTheta notationLambda notationEpsilon notationSet membership notationNot a member notationSubset or equal notationSet union notationSet intersection notationSet difference notationSet complement notationEmpty set notationCartesian product notationFor all notationThere exists notationImplies notationIf and only if notationLogical and notationLogical or notationLogical not notationSummation notationProduct notationMaximum notationArgument of the maximum notationMinimum notationArgument of the minimum notationLimit notationIntegral notationNorm notationDot product notationMatrix product notationHadamard product notationTensor product notationIdentity matrix notationZero vector notationOnes vector notationDiagonal operator notationMatrix trace notationMatrix determinant notationMatrix rank notationEigenvalue equation notationSingular value decomposition notationPositive semidefinite notationProbability notationExpectation notationStatistical independence notationConditional independence notationVariance notationStandard deviation notationCovariance notationNormal distribution notationIndependent and identically distributed notationDerivative notationPartial derivative notationGradient notationJacobian notationHessian notationBig O notationLogarithm notationExponential notationSigmoid notationSoftmax notationEntropy notationCross-entropy notationKL divergence notationIndicator function notationJoint distribution notationMarginalization notationProbability density notationCumulative distribution function notationSample mean notationSample variance notationCovariance matrix notationCorrelation coefficient notationLikelihood notationLog-likelihood notationMaximum-likelihood estimate notationMaximum a posteriori estimate notationMutual information notationConditional entropy notationPerplexity notationDifferential notationTotal derivative notationDirectional derivative notationChain rule notationJacobian-vector product notationVector-Jacobian product notationStop gradient notationTaylor expansion notationFinite difference notationMomentum estimate notationExponential decay coefficient notationAdam bias correction notationL1 regularization notationSquared L2 regularization notationSubject to notationLagrangian notationClipping notationQuery projection notationKey projection notationValue projection notationAttention mask notationCausal attention mask notationTemperature notationLayer normalization notationRMS normalization notationResidual connection notationMulti-head concatenation notationPositional encoding notationRotary position embedding notationFloating-point operation count notationPeak compute throughput notationMemory bandwidth notationArithmetic intensity notationBytes per element notationSequence length notationBatch size notationKey-value head count notationTime to first token notationTime per output token notationToken throughput notationData parallelism notationTensor parallelism notationPipeline parallelism notationAll-reduce collective notationReduce-scatter collective notationAll-gather collective notationModel FLOP utilization notationQuantization bit width notationNegative log-likelihood notationBinary cross-entropy notationMean squared error notationLog-sum-exp notationDiscounted return notationDiscount factor notationPolicy notationTransition kernel notationTrajectory notationAction-value function notationAdvantage function notationBellman expectation equation notationBellman optimality equation notationTemporal-difference error notationPolicy gradient notationPolicy importance ratio notationPPO ratio clipping notationPolicy entropy bonus notationReference policy notationPreference pair notationDPO implicit-reward logit notationGroup-normalized advantage notationAutoregressive factorization notationDiffusion noise schedule notationAlpha learning rate notationEvidence lower bound notationApproximate posterior notationPrior distribution notationLatent variable notationReparameterization trick notationReconstruction term notationDiffusion forward kernel notationCumulative diffusion signal product notationDiffusion noise prediction notationDistribution score function notationReverse diffusion kernel notationGAN minimax objective notationGenerator network notationDiscriminator network notationMarginal evidence notationAffine layer notationBias term notationActivation function notationRectified linear unit notationGaussian error linear unit notationHyperbolic tangent notationBatch normalization notationDropout mask notationConvolution notationPooling notationEmbedding lookup notationOne-hot vector notationTransformer feed-forward network notationLogits notationHidden state notationLoss function notationWeight matrix notationBias vector notationTensor shape declaration notationRow vector notationColumn vector notationTensor axis notationTensor reshape notationAxis permutation notationSqueeze axis notationUnsqueeze axis notationBroadcasting notationTensor concatenation notationTensor stacking notationTensor slice notationEllipsis notationEinstein summation expression notationContraction index notationEinstein repeated index notationKronecker delta notationCommunication time notationServing request latency notationPrefill phase notationAutoregressive decode phase notationParameter memory notationActivation memory notationOptimizer-state memory notationGradient accumulation notationMicrobatch size notationPipeline bubble notationAlpha-beta communication model notationKV-cache utilization notationTheorem statement notationLemma statement notationProposition statement notationCorollary notationProof marker notationEnd-of-proof square notationAssumed premise notationLocal let binding notationContradiction symbol notationProof by contradiction notationMathematical induction notationInduction base case notationInductive hypothesis notationInductive step notationNecessary condition notationSufficient condition notationWithout loss of generality notationSemantic entailment notationSyntactic derivability notationTherefore notationArbitrary element choice notationExists uniquely notationRoPE rotation matrix notationRoPE angular frequency notationRoPE relative-position identity notationMulti-query attention notationGrouped-query attention notationQuery-head count notationGQA query-group size notationAttention head dimension notationQuery-to-KV head mapping notationAttention score matrix notationAttention weight matrix notationAttention output projection notationSliding-window attention notationTop-k selection notationMixture-of-experts layer notationMoE router logits notationExpert routing probability notationExpert capacity notationMoE load-balancing loss notationVAE posterior mean notationVAE posterior log-variance notationStandard-normal latent noise notationPathwise derivative estimator notationELBO inference gap notationDiffusion timestep notationDiffusion noise-prediction loss notationDiffusion velocity prediction notationDiffusion clean-sample prediction notationDiffusion signal-to-noise ratio notationGAN discriminator loss notationNon-saturating GAN generator loss notationWGAN gradient penalty notationScore-matching loss notationQR decomposition notationCholesky factorization notationEigendecomposition notationPrincipal component projection notationLow-rank matrix factorization notationMoore-Penrose pseudoinverse notationOrthogonal matrix notationSymmetric matrix notationSingular value notationMatrix spectral norm notationMatrix condition number notationEmpirical risk notationEstimator bias notationUnbiased estimator notationConsistent estimator notationStandard error notationConfidence interval notationMonte Carlo estimator notationDirect preference optimization loss notationBradley-Terry preference model notationPreference probability notationGeneralized advantage estimation notationGAE trace parameter notationTemporal-difference target notationN-step return notationValue bootstrapping notationActor-critic pair notationValue-function loss notationPolicy KL penalty notationReward model score notationPairwise reward-model loss notationClipped value loss notationEligibility trace notationPolicy trust-region constraint notationPolicy value-advantage decomposition notationTensor-parallel weight shard notationRow-parallel linear layer notationColumn-parallel linear layer notationCommunication volume notationAll-to-all collective notationPoint-to-point communication notationInterconnect bandwidth notationNetwork latency notationQuantization scale notationQuantization zero-point notationDequantization notationSymmetric quantization notationQuantization group size notationQuantized parameter memory notationTight asymptotic bound notationStrict asymptotic upper order notationAsymptotic lower bound notationStrict asymptotic lower order notationDirect proof pattern notationProof by contrapositive notationProof by cases notationCounterexample notationBiconditional proof pattern notationConstructive existence proof notationStar notationTilde notationInverse superscript notationWorked formulasState value under a policy decodedScaled dot-product attention decodedGradient-descent parameter update decodedDecoder KV-cache memory decodedRoofline attainable-performance bound decodedBayes' rule decodedCategorical cross-entropy from logits decodedKullback-Leibler divergence decodedAdam parameter update decodedBellman optimality equation decodedPPO clipped surrogate objective decodedRing all-reduce communication volume decodedAutoregressive sequence log-likelihood decodedVariational autoencoder ELBO decodedDiffusion forward-process marginal decodedMulti-head attention shape flow decodedLayerNorm and RMSNorm decodedLLM serving latency decomposition decodedDirect preference optimization loss decodedVAE reparameterized latent sample decodedDiffusion noise-prediction objective decodedRoPE relative-position rotation decodedTop-k mixture-of-experts routing decodedRow-parallel tensor communication decoded
