Writing study: Exact Gaussian Moment Matching for Residual Networks: a Second-Order Method

NeurIPS overviewCorpus atlasBy contribution typeVisual galleryRules & templateWriting patternsPaper studies

← NeurIPS 2026 oral writing corpus

Simon Kuang, Xinfan Lin · Official NeurIPS 2026 oral record

Observed writing choices, not acceptance rules. This study examines the inspected manuscript version. It does not establish why the paper was accepted or that its presentation choices should be copied unchanged.

Figure, table and section ledger

Editorial contribution type: Theory & formal analysis (primary).

Layout extraction, not a required paper template. Counts are distinct caption labels; a multi-panel figure counts once. Section spans are estimated from heading positions, including floats and whitespace. Method and limitations.

Canonical PDF

167 physical pages

Before references / appendix

≈ 11.09 page-equivalents

Main figures / tables

3 / 2

Attached appendix figures / tables

123 / 216

Attached appendix span

≈ 145 page-equivalents

Before references / appendix: ≈ 6.64%References / intervening material: ≈ 6.53%Attached appendix: ≈ 86.83%Estimated layout envelopes, not text/artwork area. Unresolved appendix boundaries remain within the undivided post-body material.

Source-bound correction: Top-level heading and reference/appendix boundaries checked against source PDF text. Bibliography titles/author initials, numbered contribution/property bullets, diagram steps, equations and TOC entries are not substantive headings. Section spans are page-coordinate estimates, not ink area or prose-only length. This is an author manuscript mirror, not a verified accepted-version template. Source layout uses physical single-column y/page-height, including equation-heavy pages. Appendix span includes its explicit cover/contents/list-of-figures frontmatter; actual lettered subsection boundaries use their substantive starts, not TOC links. The large appendix contains genuinely programmatically generated cases: 123 figure captions and 216 table captions; first/last test-case tables visually checked, not all 216 individually.

Section-by-section lengths and source pages

The title/abstract/front matter precede the first numbered heading. A span ends at the next detected top-level heading, not at the last sentence; end-page whitespace is included. References and statements are listed separately. These are not measurements of text-only or artwork area.

Top-level heading	Starts on PDF page	Estimated span	Share of entire PDF
1 Introduction	1	≈ 1.93 pp.	1.16%
2 Exact, closed-form analytical expressions for mean and covariance	3	≈ 2.02 pp.	1.21%
3 Explicit examples	5	≈ 0.71 pp.	0.43%
4 Second-order accuracy of moment matching	6	≈ 1.23 pp.	0.74%
5 Examples, applications, and extensions	7	≈ 2.26 pp.	1.35%
6 Limitations	9	≈ 0.11 pp.	0.07%
7 Novelty and significance	9	≈ 1.27 pp.	0.76%
Reproducibility statement	11	≈ 1 pp.	0.6%
References	12	≈ 4 pp.	2.4%
Supplementary Material	16	≈ 8 pp.	4.79%
A Baselines	24	≈ 0.62 pp.	0.37%
B Preliminaries: Gaussian integrals	24	≈ 3.58 pp.	2.14%
C Derivation of uncertainty propagation formulas for probit activation	28	≈ 0.8 pp.	0.48%
D Derivation of uncertainty propagation formulas for GeLU activation	29	≈ 3.28 pp.	1.96%
E Derivation of uncertainty propagation formulas for ReLU activation	32	≈ 1.93 pp.	1.16%
F Derivation of uncertainty propagation formulas for Heaviside activation	34	≈ 0.79 pp.	0.47%
G Derivation of uncertainty propagation formulas for sine activation	35	≈ 2 pp.	1.2%
H Theoretical Guarantees	37	≈ 2 pp.	1.2%
I Supplement to Fig. 4.1	39	≈ 1 pp.	0.6%
J Supplement to §5.2	40	≈ 7.24 pp.	4.34%
K Supplement to §5.3	47	≈ 0.76 pp.	0.46%
L Supplement to §5.1	48	≈ 112 pp.	67.07%
M Comments on computational complexity	160	≈ 1 pp.	0.6%
NeurIPS Paper Checklist	161	≈ 6.91 pp.	4.14%

All detected figure and table locations

Main text

• Table 1.1 · p. 2
• Table 1.2 · p. 2
• Figure 4.1 · p. 7
• Figure 5.1 · p. 8
• Figure 5.2 · p. 9

Attached appendix

• Figure J.1 · p. 41
• Figure J.2 · p. 42
• Figure J.3 · p. 42
• Figure J.4 · p. 43
• Figure J.5 · p. 43
• Figure J.6 · p. 44
• Figure J.7 · p. 44
• Figure J.8 · p. 45
• Figure J.9 · p. 45
• Figure J.10 · p. 46
• Figure J.11 · p. 46
• Figure K.1 · p. 47
• Figure L.1 · p. 50
• Figure L.2 · p. 50
• Figure L.3 · p. 51
• Table L.1 · p. 52
• Table L.2 · p. 52
• Figure L.4 · p. 52
• Table L.3 · p. 53
• Table L.4 · p. 53
• Figure L.5 · p. 53
• Table L.5 · p. 54
• Table L.6 · p. 54
• Figure L.6 · p. 54
• Table L.7 · p. 55
• Table L.8 · p. 55
• Figure L.7 · p. 55
• Table L.9 · p. 56
• Table L.10 · p. 56
• Figure L.8 · p. 56
• Table L.11 · p. 57
• Table L.12 · p. 57
• Figure L.9 · p. 57
• Table L.13 · p. 58
• Table L.14 · p. 58
• Figure L.10 · p. 58
• Table L.15 · p. 59
• Table L.16 · p. 59
• Figure L.11 · p. 59
• Table L.17 · p. 60
• Table L.18 · p. 60
• Figure L.12 · p. 60
• Table L.19 · p. 61
• Table L.20 · p. 61
• Figure L.13 · p. 61
• Table L.21 · p. 62
• Table L.22 · p. 62
• Figure L.14 · p. 62
• Table L.23 · p. 63
• Table L.24 · p. 63
• Figure L.15 · p. 63
• Table L.25 · p. 64
• Table L.26 · p. 64
• Figure L.16 · p. 64
• Table L.27 · p. 65
• Table L.28 · p. 65
• Figure L.17 · p. 65
• Table L.29 · p. 66
• Table L.30 · p. 66
• Figure L.18 · p. 66
• Table L.31 · p. 67
• Table L.32 · p. 67
• Figure L.19 · p. 67
• Table L.33 · p. 68
• Table L.34 · p. 68
• Figure L.20 · p. 68
• Table L.35 · p. 69
• Table L.36 · p. 69
• Figure L.21 · p. 69
• Table L.37 · p. 70
• Table L.38 · p. 70
• Figure L.22 · p. 70
• Table L.39 · p. 71
• Table L.40 · p. 71
• Figure L.23 · p. 71
• Table L.41 · p. 72
• Table L.42 · p. 72
• Figure L.24 · p. 72
• Table L.43 · p. 73
• Table L.44 · p. 73
• Figure L.25 · p. 73
• Table L.45 · p. 74
• Table L.46 · p. 74
• Figure L.26 · p. 74
• Table L.47 · p. 75
• Table L.48 · p. 75
• Figure L.27 · p. 75
• Table L.49 · p. 76
• Table L.50 · p. 76
• Figure L.28 · p. 76
• Table L.51 · p. 77
• Table L.52 · p. 77
• Figure L.29 · p. 77
• Table L.53 · p. 78
• Table L.54 · p. 78
• Figure L.30 · p. 78
• Table L.55 · p. 79
• Table L.56 · p. 79
• Figure L.31 · p. 79
• Table L.57 · p. 80
• Table L.58 · p. 80
• Figure L.32 · p. 80
• Table L.59 · p. 81
• Table L.60 · p. 81
• Figure L.33 · p. 81
• Table L.61 · p. 82
• Table L.62 · p. 82
• Figure L.34 · p. 82
• Table L.63 · p. 83
• Table L.64 · p. 83
• Figure L.35 · p. 83
• Table L.65 · p. 84
• Table L.66 · p. 84
• Figure L.36 · p. 84
• Table L.67 · p. 85
• Table L.68 · p. 85
• Figure L.37 · p. 85
• Table L.69 · p. 86
• Table L.70 · p. 86
• Figure L.38 · p. 86
• Table L.71 · p. 87
• Table L.72 · p. 87
• Figure L.39 · p. 87
• Table L.73 · p. 88
• Table L.74 · p. 88
• Figure L.40 · p. 88
• Table L.75 · p. 89
• Table L.76 · p. 89
• Figure L.41 · p. 89
• Table L.77 · p. 90
• Table L.78 · p. 90
• Figure L.42 · p. 90
• Table L.79 · p. 91
• Table L.80 · p. 91
• Figure L.43 · p. 91
• Table L.81 · p. 92
• Table L.82 · p. 92
• Figure L.44 · p. 92
• Table L.83 · p. 93
• Table L.84 · p. 93
• Figure L.45 · p. 93
• Table L.85 · p. 94
• Table L.86 · p. 94
• Figure L.46 · p. 94
• Table L.87 · p. 95
• Table L.88 · p. 95
• Figure L.47 · p. 95
• Table L.89 · p. 96
• Table L.90 · p. 96
• Figure L.48 · p. 96
• Table L.91 · p. 97
• Table L.92 · p. 97
• Figure L.49 · p. 97
• Table L.93 · p. 98
• Table L.94 · p. 98
• Figure L.50 · p. 98
• Table L.95 · p. 99
• Table L.96 · p. 99
• Figure L.51 · p. 99
• Table L.97 · p. 100
• Table L.98 · p. 100
• Figure L.52 · p. 100
• Table L.99 · p. 101
• Table L.100 · p. 101
• Figure L.53 · p. 101
• Table L.101 · p. 102
• Table L.102 · p. 102
• Figure L.54 · p. 102
• Table L.103 · p. 103
• Table L.104 · p. 103
• Figure L.55 · p. 103
• Table L.105 · p. 104
• Table L.106 · p. 104
• Figure L.56 · p. 104
• Table L.107 · p. 105
• Table L.108 · p. 105
• Figure L.57 · p. 105
• Table L.109 · p. 106
• Table L.110 · p. 106
• Figure L.58 · p. 106
• Table L.111 · p. 107
• Table L.112 · p. 107
• Figure L.59 · p. 107
• Table L.113 · p. 108
• Table L.114 · p. 108
• Figure L.60 · p. 108
• Table L.115 · p. 109
• Table L.116 · p. 109
• Figure L.61 · p. 109
• Table L.117 · p. 110
• Table L.118 · p. 110
• Figure L.62 · p. 110
• Table L.119 · p. 111
• Table L.120 · p. 111
• Figure L.63 · p. 111
• Table L.121 · p. 112
• Table L.122 · p. 112
• Figure L.64 · p. 112
• Table L.123 · p. 113
• Table L.124 · p. 113
• Figure L.65 · p. 113
• Table L.125 · p. 114
• Table L.126 · p. 114
• Figure L.66 · p. 114
• Table L.127 · p. 115
• Table L.128 · p. 115
• Figure L.67 · p. 115
• Table L.129 · p. 116
• Table L.130 · p. 116
• Figure L.68 · p. 116
• Table L.131 · p. 117
• Table L.132 · p. 117
• Figure L.69 · p. 117
• Table L.133 · p. 118
• Table L.134 · p. 118
• Figure L.70 · p. 118
• Table L.135 · p. 119
• Table L.136 · p. 119
• Figure L.71 · p. 119
• Table L.137 · p. 120
• Table L.138 · p. 120
• Figure L.72 · p. 120
• Table L.139 · p. 121
• Table L.140 · p. 121
• Figure L.73 · p. 121
• Table L.141 · p. 122
• Table L.142 · p. 122
• Figure L.74 · p. 122
• Table L.143 · p. 123
• Table L.144 · p. 123
• Figure L.75 · p. 123
• Table L.145 · p. 124
• Table L.146 · p. 124
• Figure L.76 · p. 124
• Table L.147 · p. 125
• Table L.148 · p. 125
• Figure L.77 · p. 125
• Table L.149 · p. 126
• Table L.150 · p. 126
• Figure L.78 · p. 126
• Table L.151 · p. 127
• Table L.152 · p. 127
• Figure L.79 · p. 127
• Table L.153 · p. 128
• Table L.154 · p. 128
• Figure L.80 · p. 128
• Table L.155 · p. 129
• Table L.156 · p. 129
• Figure L.81 · p. 129
• Table L.157 · p. 130
• Table L.158 · p. 130
• Figure L.82 · p. 130
• Table L.159 · p. 131
• Table L.160 · p. 131
• Figure L.83 · p. 131
• Table L.161 · p. 132
• Table L.162 · p. 132
• Figure L.84 · p. 132
• Table L.163 · p. 133
• Table L.164 · p. 133
• Figure L.85 · p. 133
• Table L.165 · p. 134
• Table L.166 · p. 134
• Figure L.86 · p. 134
• Table L.167 · p. 135
• Table L.168 · p. 135
• Figure L.87 · p. 135
• Table L.169 · p. 136
• Table L.170 · p. 136
• Figure L.88 · p. 136
• Table L.171 · p. 137
• Table L.172 · p. 137
• Figure L.89 · p. 137
• Table L.173 · p. 138
• Table L.174 · p. 138
• Figure L.90 · p. 138
• Table L.175 · p. 139
• Table L.176 · p. 139
• Figure L.91 · p. 139
• Table L.177 · p. 140
• Table L.178 · p. 140
• Figure L.92 · p. 140
• Table L.179 · p. 141
• Table L.180 · p. 141
• Figure L.93 · p. 141
• Table L.181 · p. 142
• Table L.182 · p. 142
• Figure L.94 · p. 142
• Table L.183 · p. 143
• Table L.184 · p. 143
• Figure L.95 · p. 143
• Table L.185 · p. 144
• Table L.186 · p. 144
• Figure L.96 · p. 144
• Table L.187 · p. 145
• Table L.188 · p. 145
• Figure L.97 · p. 145
• Table L.189 · p. 146
• Table L.190 · p. 146
• Figure L.98 · p. 146
• Table L.191 · p. 147
• Table L.192 · p. 147
• Figure L.99 · p. 147
• Table L.193 · p. 148
• Table L.194 · p. 148
• Figure L.100 · p. 148
• Table L.195 · p. 149
• Table L.196 · p. 149
• Figure L.101 · p. 149
• Table L.197 · p. 150
• Table L.198 · p. 150
• Figure L.102 · p. 150
• Table L.199 · p. 151
• Table L.200 · p. 151
• Figure L.103 · p. 151
• Table L.201 · p. 152
• Table L.202 · p. 152
• Figure L.104 · p. 152
• Table L.203 · p. 153
• Table L.204 · p. 153
• Figure L.105 · p. 153
• Table L.205 · p. 154
• Table L.206 · p. 154
• Figure L.106 · p. 154
• Table L.207 · p. 155
• Table L.208 · p. 155
• Figure L.107 · p. 155
• Table L.209 · p. 156
• Table L.210 · p. 156
• Figure L.108 · p. 156
• Table L.211 · p. 157
• Table L.212 · p. 157
• Figure L.109 · p. 157
• Table L.213 · p. 158
• Table L.214 · p. 158
• Figure L.110 · p. 158
• Table L.215 · p. 159
• Table L.216 · p. 159
• Figure L.111 · p. 159

Argument and organization

1. Problem and assumptions comparison

   Locates the mathematical gap by comparing restrictions in earlier moment propagation methods.

2. Closed-form moments and accuracy guarantees

   Separates exact layerwise formulas from the approximate distribution propagated through a deep network.

3. Experiments and derivation appendices

   Tests approximation quality and organizes detailed derivations by activation function and application.

Problem, claim and contribution

Two related-work tables compare assumptions and supported activations. The gap is a precise mathematical restriction, rather than an unsupported claim that prior uncertainty propagation is generally inadequate.

Introduction; Tables 1.1–1.2 · Main paper

Argument and section order

Definitions of the analytic approximation and its target precede explicit examples and accuracy theorems. Experiments then test approximation quality, keeping the logical dependence between claim, guarantee and numerical check visible.

Sections 2–5 · Main paper

Figures, captions and visual explanation

Paired box plots separate small- and large-input-variance settings on logarithmic error axes. The visual shows a distribution across networks rather than a single best case; its caption states that lower KL is better.

Figure 5.1, PDF p. 8 · Main paper

Tables, comparisons and evidence

Assumption tables are conceptual evidence, not accuracy leaderboards. The Bayesian-network experiment separately explains why end-to-end predictive quality is not a direct measurement of distributional approximation accuracy.

Tables 1.1–1.2, PDF p. 2; Section 5.2 · Main paper

Appendix and reproducibility

Activation-specific derivations, baseline definitions and theoretical guarantees have named destinations; long later result sections support the numerical comparisons. This was a writing/navigation scan, not a verification of every proof or result in the 167-page source.

Appendices A–H and later experimental appendices · Main paper

Clarity and writing conventions

The wording distinguishes exact moments at a layer from approximation of the full output distribution. Maintaining that distinction in theorem statements, abstract and captions prevents a mathematically stronger claim than the analysis supports.

Section 4; Section 5.2 · Main paper

What to reuse—and what not to infer

Transferable lesson: Use a compact assumptions table to establish the theoretical gap, then distinguish exact identities, approximation guarantees and empirical evidence.

Boundary: This long public arXiv revision is not verified identical to the accepted manuscript. Neither its appendix length nor visible citation-box styling is a conference formatting recommendation.

Apply this lesson using the argument and evidence worksheet.

Inspection record and sources

Checked

2026-10-02

Reading status

Writing-focused manuscript scan + selected visuals inspected

Main text

Writing-focused scan across framing, method organization, captions, result presentation and conclusion; title and author overlap checked against the official program. Selected physical pages inspected as rendered images.

Appendix / supplement

Attached appendix headings, reproduction routes and representative supporting prose/captions scanned. Long tables, every proof and every appendix page were not individually validated.

Visually inspected PDF pages

2, 8

This is a writing-focused scan of the manuscript and the stated supplementary scope: framing, organization, visual explanation, evidence presentation and wording. It is not a line-by-line proof audit, reproduction of results or exhaustive scientific review.

• Public author manuscript arXiv:2601.22307v2; not verified identical to the accepted OpenReview revision · 167 pages