The ambient derivative transpose — analysis & deferral note#

Status: DEFERRED (not currently needed). The symmetric-derivative work (branch probe-derivatives) mirrors the probing surface with derivative versions for the forward, tangent (Riemannian), and corewise (non-manifold core-gradient) flavors of probe/apply/entries. The ambient flavor — the frame-free adjoint on the full tensor space, returning canonical (CP) factors — was the one cell of the grid left unbuilt. This note records the math and the design analysis so it can be picked back up if a use case arises. It is not scheduled work.

Companion docs: docs/transposes.md (the ambient/corewise/tangent taxonomy), docs/symmetric_probe_derivatives.tex (the forward + tangent-transpose math), dev/archive/derivatives_mirror_plan.md (the overall plan/handoff). Backend home if built: backend/sampling_derivatives.py; the non-derivative versions to mirror are probing.t3_probe_ambient_transpose, apply.t3_apply_ambient_transpose, entries.t3_entries_ambient_transpose.

1. Recap: the non-derivative ambient transposes#

The ambient transpose is the literal adjoint of a sampling op viewed as a linear map on the full tensor space T ↦ samples. Its output is the back-projection tensor, returned as CP factors (the natural type — a multilinear adjoint emits one vector per mode), realized via from_canonical:

  • apply (T ↦ ⟨T, w₀⊗…⊗w_{d-1}⟩, a scalar): adjoint of residual c is c·(w₀⊗…⊗w_{d-1}) — rank-1 CP.

  • probe (T ↦ {⟨T, w₀⊗…⊗·⊗…⊗w_{d-1}⟩}ᵢ, d vectors): adjoint of {ztildeᵢ} is Σᵢ w₀⊗…⊗ztildeᵢ⊗…⊗w_{d-1} (residual in slot i, probe vectors elsewhere) — rank-d CP.

  • entries = apply with one-hot vectors e_{index}: a rank-1 one-hot scatter. Needs an explicit shape (the one-hots don’t carry the ambient dims Nᵢ).

sum_over_probes=True folds the probe stack W into the CP rank (one rank-d|W| / |W| CP); False keeps W as a stacking axis.

2. The derivative ambient transpose: the math#

The derivative forward is the jet (in s) of the plain forward along X + sP. For apply, the plain forward is apply(T) = ⟨T, ⊗ⱼwⱼ⟩, so

applyᵗ(T) = dᵗ/dsᵗ ⟨T, ⊗ⱼ(wⱼ + s pⱼ)⟩|₀ = ⟨T, Mₜ⟩,   Mₜ := t!·[sᵗ] ⊗ⱼ(wⱼ + s pⱼ).

applyᵗ is linear in T, with “matrix” Mₜ. So the ambient adjoint of the residual jet {c⁽ᵗ⁾} is

Tᵀ = Σₜ c⁽ᵗ⁾ Mₜ = Σₜ c⁽ᵗ⁾ · t! · [sᵗ] ⊗ⱼ(wⱼ + s pⱼ)          (apply)

and, the same way (residual in the free slot i, the off-modes j≠i perturbed),

Tᵀ = Σᵢ Σₜ rᵢ⁽ᵗ⁾ · t! · [sᵗ] ⊗_{j≠i}(wⱼ + s pⱼ)   placed in slot i   (probe)

Reading. Tᵀ is exactly the jet of the non-derivative ambient back-projection, evaluated at the perturbed probe vectors wⱼ + s pⱼ, then contracted against the residual jet on the order axis. The [sᵗ] coefficient of the product ⊗ⱼ(wⱼ+s pⱼ) is a binomial/multinomial jet convolution of the d per-mode 2-jets [wⱼ, pⱼ] (value at order 0, direction at order 1) — so the binomial tensor trs appears in the outer products, as one would expect by analogy with the forward.

entries. Apply with one-hot in the frame slot and a general direction: vectors (e_{index_j} + s pⱼ). So Tᵀ = Σₜ c⁽ᵗ⁾ t! [sᵗ] ⊗ⱼ(e_{index_j} + s pⱼ) — the w-slots become one-hot scatters at index. Unlike the non-derivative entries_ambient_transpose, no explicit shape is needed: pp[j] carries Nⱼ.

3. The catch: the back-projection is inherently high-rank#

The crucial fact. The coefficient

[sᵗ] ⊗ⱼ(wⱼ + s pⱼ) = Σ_{|S|=t} ⊗ⱼ (pⱼ if j∈S else wⱼ)

is an elementary-symmetric combination — a genuine sum of C(d-1,t) (probe) / C(d,t) (apply) rank-1 terms, not a single one. Summing over t:

op

CP rank of Tᵀ

apply

2^d

probe

d · 2^{d-1}

entries

2^d (one-hot factors)

(× |W| more if sum_over_probes=True folds W into the rank.) This is a property of the tensor itself, not of how we write it: unlike the clean rank-1 / rank-d non-derivative ambient back-projections, the derivative one is exponentially higher rank. d is usually small (2^{d-1} ≤ 32 at d=4), so it is finite and computable — but the materialized object (and any from_canonical T3 built from it, whose ranks track the CP rank) is intrinsically large.

4. Two representations (neither removes the rank)#

  • (A) Subset-expanded standard CP. Enumerate S (which modes contribute p vs w); the residual enters at order |S| in the free slot, weighted by |S|!. Yields plain CP factors that from_canonical already consumes — zero new machinery, a ~10-line nested loop over i and the 2^{d-1} subsets. Rank d·2^{d-1} is intrinsic anyway. Recommended if built.

  • (B) Jet-CP (the “trs in the outer products” idea). Keep d per-mode 2-jet factors [wⱼ,pⱼ] + the residual jet, and convolve the orders via a d-fold multinomial trs (a trsq…q tensor) only at reconstruction. Compact representation (rank d/1), but needs a new jet-CP type + a d-fold multinomial reconstruction, and it still materializes to the same d·2^{d-1} rank. So (B) defers the blow-up without removing it.

5. Why deferred#

  • Least-used flavor. Derivative-fitting uses the tangent transpose (Riemannian) or corewise (Adam/L-BFGS), both done. The ambient one exists to form a dense Euclidean gradient as a CP sum — rarely the thing you want, and here that sum is exponential-rank rather than the clean rank-d the non-derivative version gives.

  • Output inherently less clean than its non-derivative sibling — the one place the strict-generalization stays ugly regardless of representation.

  • No concrete use case; added (if ever) for grid completeness only.

6. Implementation sketch if picked up (option A)#

Mirror probing.t3_probe_ambient_transpose (which builds the rank-d CP with the residual on a diagonal slot), but enumerate subsets:

  • apply_ambient_derivatives_transpose(c, ww, pp, order, sum_over_probes=False) → CP rank 2^d: for each subset S ⊆ {0..d-1}, a term with factor-k = pp[k] if k∈S else ww[k], scaled (fold into one factor) by c[|S|]·|S|!. Only subsets with |S| ≤ order contribute (residual has orders 0..order); above the tensor order the derivative vanishes anyway.

  • probe_ambient_derivatives_transpose(ztildes, ww, pp, order, sum_over_probes=False) → CP rank d·2^{d-1}: for each mode i and each S ⊆ {j≠i}, factor-i = ztildes[i][|S|], factor-k (k≠i) = pp[k] if k∈S else ww[k], scaled by |S|!.

  • entries_ambient_derivatives_transpose(c, index, pp, order, sum_over_probes=False) → as apply with ww[k] = e_{index_k} (one-hot, length Nₖ from pp[k]); no explicit shape arg.

  • Frontend: TuckerTensorTrain.{probe,apply,entries}_ambient_derivatives_transpose (static), returning CP factors for from_canonical, paralleling the non-derivative *_ambient_transpose statics.

  • Verify: the dense adjoint identity ⟨Tᵀ, X⟩_F == Σ ⟨residual, sample_derivatives(X)⟩ (from_canonical the factors, compare to the forward derivative of a random X). Log the CP rank (no silent caps).

  • sum_over_probes: fold W into the rank exactly as the non-derivative versions do.