Lecture 15: Exact Tensor Completion Joint Work with David Steurer - - PowerPoint PPT Presentation

β–Ά
lecture 15 exact tensor completion
SMART_READER_LITE
LIVE PREVIEW

Lecture 15: Exact Tensor Completion Joint Work with David Steurer - - PowerPoint PPT Presentation

Sep 05, 2023 205 likes β€’681 views

Lecture 15: Exact Tensor Completion Joint Work with David Steurer Lecture Outline Part I: Matrix Completion Problem Part II: Matrix Completion via Nuclear Norm Minimization Part III: Generalization to Tensor Completion

slide-1
SLIDE 1

Lecture 15: Exact Tensor Completion

Joint Work with David Steurer

slide-2
SLIDE 2

Lecture Outline

Part I: Matrix Completion Problem

  • Part II:
  • Matrix Completion via Nuclear Norm

Minimization Part III:

  • Generalization to Tensor Completion

Part IV: SOS

  • symmetry to the Rescue

Part

  • V: Finding Dual Certificate for Matrix

Completion Part

  • VI: Open Problems
slide-3
SLIDE 3

Part I: Matrix Completion Problem

slide-4
SLIDE 4

Matrix Completion

  • Matrix Completion: Let Ξ© be a set of entries

sampled at random. Given the entries {𝑁𝑏𝑐: 𝑏, 𝑐 ∈ Ξ©} from a matrix 𝑁, can we determine the remaining entries of 𝑁?

  • Impossible in general, tractable if 𝑁 is low rank

i.e. 𝑁 = σ𝑗=1

𝑠

πœ‡π‘—π‘£π‘—π‘€π‘—

π‘ˆ where 𝑠 is not too large.

slide-5
SLIDE 5

Netflix Challenge

  • Canonical example of matrix completion:

Netflix Challenge

  • Can we predict users’ preferences on other

movies from their previous ratings?

slide-6
SLIDE 6

Netflix Challenge

10 9 8 9 6 7.5 6 6 5 8 9 9 ? ? ? ? ? ? ? ?

slide-7
SLIDE 7

Solving Matrix Completion

  • Current best method in practice: Alternating

minimization

  • Idea: Write 𝑁 = σ𝑗=1

𝑠

𝑣𝑗 𝑀𝑗

π‘ˆ, alternate

between optimizing {𝑣𝑗} and {𝑀𝑗}

  • Best known theoretical guarantees: Nuclear

norm minimization

  • This lecture: We’ll describe nuclear norm

minimization and how it generalizes to tensor completion via SOS.

slide-8
SLIDE 8

Part II: Nuclear Norm Minimization

slide-9
SLIDE 9

Theorem Statement

  • Theorem [Rec11]: If 𝑁 = σ𝑗=1

𝑠

πœ‡π‘— 𝑣𝑗𝑀𝑗

π‘ˆ is an

π‘œ Γ— π‘œ matrix then nuclear norm minimization requires 𝑃(π‘œπ‘ πœˆ0 π‘šπ‘π‘•π‘œ 2) random samples to complete 𝑁 with high probability

  • Note: 𝜈0 is a parameter related to how

coherent the {𝑣𝑗} and the {𝑀𝑗} (see appendix for the definition)

  • Example of why this is needed: If 𝑣𝑗 = 𝑓

π‘˜ then

𝑣𝑗𝑀𝑗

π‘ˆ = π‘“π‘˜π‘€π‘— π‘ˆ can only be fully detected by

sampling all of row π‘˜, which requires sampling almost everything!

slide-10
SLIDE 10

Nuclear Norm

  • Recall the singular value decomposition (SVD)
  • f a matrix 𝑁
  • 𝑁 = σ𝑗=1

𝑠

πœ‡π‘—π‘£π‘— 𝑀𝑗

π‘ˆ where the {𝑣𝑗} are

  • rthonormal, the {𝑀𝑗} are orthonormal, and

πœ‡π‘— β‰₯ 0 for all 𝑗.

  • The nuclear norm of 𝑁 is 𝑁 βˆ— = σ𝑗=1

𝑠

πœ‡π‘—

slide-11
SLIDE 11

Nuclear Norm Minimization

  • Matrix completion problem: Recover 𝑁 given

randomly sampled entries {𝑁𝑏𝑐: 𝑏, 𝑐 ∈ Ξ©}

  • Nuclear norm minimization: Find the matrix

π‘Œ which minimizes π‘Œ βˆ— while satisfying π‘Œπ‘π‘ = 𝑁𝑏𝑐 whenever 𝑏, 𝑐 ∈ Ξ©.

  • How do we minimize π‘Œ βˆ—?
slide-12
SLIDE 12

Semidefinite Program

  • We can implement nuclear norm minimization

with the following semidefinite program:

  • Minimize the trace of 𝑉

π‘Œ π‘Œπ‘ˆ π‘Š ≽ 0 where π‘Œπ‘π‘ = 𝑁𝑏𝑐 whenever 𝑏, 𝑐 ∈ Ξ©

  • Why does this work? We’ll first show that the

true solution is a good solution. We’ll then describe how to show the true solution is the

  • ptimal solution
slide-13
SLIDE 13

True Solution

  • Program: Minimize the trace of 𝑉

π‘Œ π‘Œπ‘ˆ π‘Š ≽ 0 where π‘Œπ‘π‘ = 𝑁𝑏𝑐 whenever 𝑏, 𝑐 ∈ Ξ©

  • True solution: 𝑉

π‘Œ π‘Œπ‘ˆ π‘Š = σ𝑗 πœ‡π‘— 𝑣𝑗 𝑀𝑗 𝑣𝑗

π‘ˆ

𝑀𝑗

π‘ˆ

(recall that 𝑁 = σ𝑗 πœ‡π‘— 𝑣𝑗𝑀𝑗

π‘ˆ)

  • Since for all 𝑗, 𝑒𝑠 𝑣𝑗𝑣𝑗

π‘ˆ = 𝑒𝑠 𝑀𝑗𝑀𝑗 π‘ˆ = 1,

𝑒𝑠 𝑉 π‘Œ π‘Œπ‘ˆ π‘Š = 2 σ𝑗 πœ‡π‘—

slide-14
SLIDE 14

Dual Certificate

  • Program: Minimize the trace of 𝑉

π‘Œ π‘Œπ‘ˆ π‘Š ≽ 0 where π‘Œπ‘π‘ = 𝑁𝑏𝑐 whenever 𝑏, 𝑐 ∈ Ξ©

  • Dual Certificate:

𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 ≽ 0

  • Recall that if 𝑁1, 𝑁2 ≽ 0 then 𝑁1⦁𝑁2 β‰₯ 0

(where ⦁ is the entry-wise dot product)

  • 𝐽𝑒

βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 ⦁ 𝑉 π‘Œ π‘Œπ‘ˆ π‘Š β‰₯ 0

  • If 𝐡𝑏𝑐 = 0 whenever 𝑏, 𝑐 βˆ‰ Ξ©, this lower

bounds the trace.

slide-15
SLIDE 15

True Solution Optimality

  • Dual Certificate:

𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 ≽ 0 where 𝐡𝑏𝑐 = 0 whenever 𝑏, 𝑐 βˆ‰ Ξ©

  • True solution 𝑉

π‘Œ π‘Œπ‘ˆ π‘Š = σ𝑗 πœ‡π‘— 𝑣𝑗 𝑀𝑗 𝑣𝑗

π‘ˆ

𝑀𝑗

π‘ˆ

is optimal if 𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 ⦁ 𝑉 π‘Œ π‘Œπ‘ˆ π‘Š = 0

  • This occurs if

𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 𝑣𝑗 𝑀𝑗 = 0 for all 𝑗

slide-16
SLIDE 16

Conditions on 𝐡

  • We want 𝐡 such that

𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 ≽ 0, 𝐡𝑏𝑐 = 0 whenever 𝑏, 𝑐 βˆ‰ Ξ©, and 𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 𝑣𝑗 𝑀𝑗 = 0 for all 𝑗

  • Necessary and sufficient conditions on 𝐡:

1. 𝐡 ≀ 1

  • 2. 𝐡𝑏𝑐 = 0 whenever 𝑏, 𝑐 βˆ‰ Ξ©
  • 3. 𝐡𝑀𝑗 = 𝑣𝑗 for all 𝑗
  • 4. π΅π‘ˆπ‘£π‘— = 𝑀𝑗 for all 𝑗
slide-17
SLIDE 17

Dual Certificate with all entries

  • Necessary and sufficient conditions on 𝐡:

1. 𝐡 ≀ 1

  • 2. 𝐡𝑏𝑐 = 0 whenever 𝑏, 𝑐 βˆ‰ Ξ©
  • 3. 𝐡𝑀𝑗 = 𝑣𝑗 for all 𝑗
  • 4. π΅π‘ˆπ‘£π‘— = 𝑀𝑗 for all 𝑗
  • If we have all entries (so we can ignore

condition 2), we can take 𝐡 = σ𝑗 𝑣𝑗𝑀𝑗

π‘ˆ

  • Challenge: Find 𝐡 when we don’t have all

entries

  • Remark: This explains why the semidefinite

program minimizes the nuclear norm.

slide-18
SLIDE 18

Part III: Generalization to Tensor Completion

slide-19
SLIDE 19

Tensor Completion

Tensor Completion: Let

  • Ξ© be a set of entries

sampled at random. Given the entries {π‘ˆπ‘π‘π‘‘: 𝑏, 𝑐, 𝑑 ∈ Ξ©} from a tensor π‘ˆ, can we determine the remaining entries of π‘ˆ? More difficult problem: tensor rank is much

  • more complicated
slide-20
SLIDE 20

Exact Tensor Completion Theorem

  • Theorem [PS17]: If π‘ˆ = σ𝑗=1

𝑠

πœ‡π‘—π‘£π‘— βŠ— 𝑀𝑗 βŠ— π‘₯𝑗, the {𝑣𝑗} are orthogonal, the {𝑀𝑗} are

  • rthogonal, and the {π‘₯𝑗} are orthogonal then

with high probability we can recover π‘ˆ with 𝑃(π‘ πœˆπ‘œ

3 2π‘žπ‘π‘šπ‘§π‘šπ‘π‘•(π‘œ)) random samples

  • First algorithm to obtain exact tensor

completion

  • Remark: The orthogonality condition is very

restrictive but this result can likely be extended.

  • See appendix for the definition of 𝜈.
slide-21
SLIDE 21

Semidefinite Program: First Attempt

  • Won’t quite work, but we’ll fix it later.
  • Minimize the trace of 𝑉

π‘Œ π‘Œπ‘ˆ π‘Šπ‘‹ ≽ 0 where π‘Œπ‘π‘π‘‘ = π‘ˆπ‘π‘π‘‘ whenever 𝑏, 𝑐, 𝑑 ∈ Ξ©

  • Here the top and left blocks are indexed by 𝑏

and the bottom and right blocks are indexed by 𝑐, 𝑑.

slide-22
SLIDE 22

True Solution

Program: Minimize trace of

  • 𝑉

π‘Œ π‘Œπ‘ˆ π‘Šπ‘‹ ≽ 0 where π‘Œπ‘π‘π‘‘ = π‘ˆπ‘π‘π‘‘ whenever 𝑏, 𝑐, 𝑑 ∈ Ξ© True solution:

  • 𝑉

π‘Œ π‘Œπ‘ˆ π‘Šπ‘‹ = σ𝑗 πœ‡π‘— 𝑣𝑗 𝑀𝑗 βŠ— π‘₯𝑗 𝑣𝑗

π‘ˆ

𝑀𝑗 βŠ— π‘₯𝑗 π‘ˆ (recall that T = σ𝑗 πœ‡π‘— 𝑣𝑗 𝑀𝑗 βŠ— π‘₯𝑗 π‘ˆ) 𝑒𝑠

  • 𝑉

π‘Œ π‘Œπ‘ˆ π‘Šπ‘‹ = 2 σ𝑗 πœ‡π‘—

slide-23
SLIDE 23

Dual Certificate: First Attempt

Program: Minimize trace of

  • 𝑉

π‘Œ π‘Œπ‘ˆ π‘Šπ‘‹ ≽ 0 where π‘Œπ‘π‘π‘‘ = π‘ˆπ‘π‘π‘‘ whenever 𝑏, 𝑐, 𝑑 ∈ Ξ© Dual Certificate:

  • 𝐽𝑒

βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 ≽ 0 where 𝐡𝑏𝑐𝑑 = 0 whenever 𝑏, 𝑐, 𝑑 βˆ‰ Ξ© We want

  • 𝐽𝑒

βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 𝑣𝑗 𝑀𝑗 βŠ— π‘₯𝑗 = 0 for all 𝑗

slide-24
SLIDE 24

Conditions on 𝐡

  • We want 𝐡 such that

𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 ≽ 0, 𝐡𝑏𝑐𝑑 = 0 whenever 𝑏, 𝑐, 𝑑 βˆ‰ Ξ©, and 𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ 𝐽𝑒 𝑣𝑗 𝑀𝑗 βŠ— π‘₯𝑗 = 0 for all 𝑗

  • Necessary and sufficient conditions on 𝐡:

1. 𝐡 ≀ 1

  • 2. 𝐡𝑏𝑐𝑑 = 0 whenever 𝑏, 𝑐, 𝑑 βˆ‰ Ξ©
  • 3. 𝐡(𝑀𝑗 βŠ— π‘₯𝑗) = 𝑣𝑗 for all 𝑗
  • 4. π΅π‘ˆπ‘£π‘— = 𝑀𝑗 βŠ— π‘₯𝑗 for all 𝑗 TOO STRONG, requires

Ξ©(π‘œ2) samples!

slide-25
SLIDE 25

Part IV: SOS-symmetry to the Rescue

slide-26
SLIDE 26

SOS Program

  • Minimize the trace of 𝑉

π‘Œ π‘Œπ‘ˆ π‘Šπ‘‹ ≽ 0 where π‘Œπ‘π‘π‘‘ = π‘ˆπ‘π‘π‘‘ whenever 𝑏, 𝑐, 𝑑 ∈ Ξ© and π‘Šπ‘‹ is SOS-symmetric (i.e. π‘Šπ‘‹π‘π‘‘π‘β€²π‘‘β€² = π‘Šπ‘‹π‘β€²π‘‘π‘π‘‘β€² for all 𝑐, 𝑑, 𝑐′, 𝑑′)

slide-27
SLIDE 27

Review: Matrix Polynomial π‘Ÿ(𝑅)

  • Definition: Given a symmetric matrix 𝑅

indexed by monomials, define q 𝑅 = σ𝐿(σ𝐽,𝐾:𝐿=𝐽βˆͺ𝐾(𝑏𝑑 π‘›π‘£π‘šπ‘’π‘—π‘‘π‘“π‘’π‘‘) 𝑅𝐽𝐾)𝑦𝐿

  • Idea: M βˆ™ 𝑅 = ΰ·¨

𝐹[π‘Ÿ(𝑅)]

slide-28
SLIDE 28

Dual Certificate

  • Program: Minimize trace of 𝑉

π‘Œ π‘Œπ‘ˆ π‘Šπ‘‹ ≽ 0 where π‘Œπ‘π‘π‘‘ = π‘ˆπ‘π‘π‘‘ whenever 𝑏, 𝑐, 𝑑 ∈ Ξ© and π‘Šπ‘‹ is SOS-symmetric

  • Dual Certificate:

𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ 𝐢 ≽ 0 where 𝐡𝑏𝑐𝑑 = 0 whenever 𝑏, 𝑐, 𝑑 βˆ‰ Ξ© and q 𝐢 = π‘Ÿ(𝐽𝑒)

  • We want

𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ 𝐢 𝑣𝑗 𝑀𝑗 βŠ— π‘₯𝑗 = 0 for all 𝑗

slide-29
SLIDE 29

Dual Certificate Tightness Condition

  • Write 𝐢 = π΅π‘ˆπ΅ + 𝐽𝑒 βˆ’ 𝑆
  • Dual Certificate:

𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ π΅π‘ˆπ΅ + 𝐽𝑒 βˆ’ 𝑆 ≽ 0 where 𝐡𝑏𝑐𝑑 = 0 whenever 𝑏, 𝑐, 𝑑 βˆ‰ Ξ© and q 𝐢 = π‘Ÿ(𝐽𝑒)

  • This dual certificate is tight for the true solution

if 𝐽𝑒 βˆ’π΅ βˆ’π΅π‘ˆ π΅π‘ˆπ΅ + 𝐽𝑒 βˆ’ 𝑆 𝑣𝑗 𝑀𝑗 βŠ— π‘₯𝑗 = 0 for all 𝑗

slide-30
SLIDE 30

Dual Certificate Conditions

  • This gives us the following conditions on 𝐡, 𝑆

1. 𝐡𝑏𝑐𝑑 = 0 whenever 𝑏, 𝑐, 𝑑 βˆ‰ Ξ© 2. βˆ€π‘—, 𝐡(π‘€π‘—βŠ— π‘₯𝑗) = 𝑣𝑗 3. 𝑆 ≀ 1 4. βˆ€π‘—, 𝑆(π‘€π‘—βŠ— π‘₯𝑗) = 𝑀𝑗 βŠ— π‘₯𝑗 5. π‘Ÿ 𝑆 = π‘Ÿ(π΅π‘ˆπ΅) (so that π‘Ÿ 𝐢 = π‘Ÿ 𝐽𝑒 = σ𝑐,𝑑 𝑧𝑐

2𝑨𝑑 2)

  • Remark: These conditions are sufficient even if

π‘ˆ is not orthogonal. We only prove the theorem for orthogonal tensors because that’s what our current analysis can handle.

slide-31
SLIDE 31

Part V: Finding Dual Certificate for Matrix Completion

slide-32
SLIDE 32

Conditions on 𝐡

Necessary and sufficient conditions on

  • 𝐡:

1. 𝐡 ≀ 1 2. 𝐡𝑏𝑐 = 0 whenever 𝑏, 𝑐 βˆ‰ Ξ© 3. 𝐡𝑀𝑗 = 𝑣𝑗 for all 𝑗 4. π΅π‘ˆπ‘£π‘— = 𝑀𝑗 for all 𝑗

How can we find such an

  • 𝐡?

Idea:

  • Alternate between satisfying condition 2

and conditions 3,4, converging to a final solution.

slide-33
SLIDE 33

Definition of P

U, P V, P T

  • Define 𝑄𝑉 to be the projection to π‘‘π‘žπ‘π‘œ 𝑣𝑗 .

The equation for this is 𝑄𝑉 𝑦 = σ𝑗 𝑦 β‹… 𝑣𝑗 𝑣𝑗

  • Define π‘„π‘Š to be the projection to π‘‘π‘žπ‘π‘œ 𝑀𝑗 .

The equation for this is π‘„π‘Š 𝑧 = σ𝑗 𝑧 β‹… 𝑀𝑗 𝑀𝑗

  • Define π‘„π‘ˆ to be the projection (on the space of

matrices) to π‘‘π‘žπ‘π‘œ{𝑦𝑀𝑗

π‘ˆ, 𝑣𝑗 π‘ˆπ‘§} (for arbitrary

𝑦, 𝑧). The equation for this is π‘„π‘ˆπ‘ = 𝑄𝑉𝑁 + π‘„π‘Šπ‘ βˆ’ π‘„π‘‰π‘π‘„π‘Š

slide-34
SLIDE 34

Restatement of Conditions 3,4

  • Necessary and sufficient conditions on 𝐡:

1. 𝐡 ≀ 1 2. 𝐡𝑏𝑐 = 0 whenever 𝑏, 𝑐 βˆ‰ Ξ© 3. 𝐡𝑀𝑗 = 𝑣𝑗 for all 𝑗 4. π΅π‘ˆπ‘£π‘— = 𝑀𝑗 for all 𝑗

  • Without loss of generality, assume 𝑁 =

σ𝑗 𝑣𝑗𝑀𝑗

π‘ˆ (the values of the πœ‡π‘— don’t affect the

dual certificate)

  • Assuming 𝑁 = σ𝑗 𝑣𝑗𝑀𝑗

π‘ˆ, conditions 3,4 are

equivalent to π‘„π‘ˆπ΅ = 𝑁

slide-35
SLIDE 35

Definition of 𝑆Ω and ΰ΄€ 𝑆Ω

Definition: Define 𝑆Ω(π‘Œ) = π‘œ1π‘œ2π‘œ3

𝑛

π‘Œπ‘π‘π‘‘ if

  • 𝑏, 𝑐, 𝑑 ∈ Ξ© and 0 otherwise where π‘œ1 Γ— π‘œ2 Γ—

π‘œ3 are the dimensions of the tensor and each entry is sampled indepently with probability

𝑛 π‘œ1π‘œ2π‘œ3.

Define

  • ΰ΄€

𝑆Ω(π‘Œ) =

π‘œ1π‘œ2π‘œ3 𝑛

βˆ’ 1 π‘Œπ‘π‘π‘‘ if 𝑏, 𝑐, 𝑑 ∈ Ξ© and βˆ’π‘Œπ‘π‘π‘‘ if 𝑏, 𝑐, 𝑑 βˆ‰ Ξ©

  • 𝑆Ω π‘Œ 𝑏𝑐𝑑 = 0 whenever 𝑏, 𝑐, 𝑑 βˆ‰ Ξ©

𝐹

  • ΰ΄€

𝑆Ω π‘Œ = 0 (over the choice of Ξ©)

slide-36
SLIDE 36

First Iteration

Start with

  • 𝑁. P

T𝑁 = 𝑁 but 𝑁 has nonzero

entries outside the sampled entries

  • 𝑆Ω(𝑁) is zero outside the sampled entries,

but π‘„π‘ˆπ‘†Ξ© 𝑁 β‰  𝑁 We take

  • A1 = 𝑆Ω(𝑁) as the first

approximation, we’ll need to correct for the difference π‘„π‘ˆπ‘†Ξ©π‘ βˆ’ 𝑁 = π‘„π‘ˆ ΰ΄€ 𝑆Ω𝑁

slide-37
SLIDE 37

Technical Note

  • For the analysis, actually need to resample

independently for each iteration, obtaining sets of samples Ξ©1, Ξ©2, …. This is the source of the π‘šπ‘π‘•π‘œ 2 in the upper bound (the lower bound only has log π‘œ (reference to be added))

slide-38
SLIDE 38

Iterative Equation

  • Take

𝐡𝑙 = Οƒπ‘˜=0

π‘™βˆ’1 βˆ’1 π‘˜π‘†Ξ©π‘˜+1(π‘„π‘ˆ ΰ΄€

π‘†Ξ©π‘˜) … (π‘„π‘ˆ ΰ΄€ 𝑆Ω1)𝑁

  • Claim:

π‘„π‘ˆπ΅π‘™ = 𝑁 + βˆ’1 π‘™βˆ’1(π‘„π‘ˆ ΰ΄€ 𝑆Ω𝑙) … (π‘„π‘ˆ ΰ΄€ 𝑆Ω1)𝑁

  • Proof idea: Use the facts that 𝑆Ω = 1 + ΰ΄€

𝑆Ω, π‘„π‘ˆ

2 = π‘„π‘ˆ, and π‘„π‘ˆπ‘ = 𝑁.

slide-39
SLIDE 39

Convergence and Final Step

Take

  • 𝐡𝑙 = Οƒπ‘˜=0

π‘™βˆ’1 βˆ’1 π‘˜π‘†Ξ©π‘˜+1(π‘„π‘ˆ ΰ΄€

π‘†Ξ©π‘˜) … (π‘„π‘ˆ ΰ΄€ 𝑆Ω1)𝑁 Claim:

  • π‘„π‘ˆπ΅π‘™ = 𝑁 + βˆ’1 π‘™βˆ’1(π‘„π‘ˆ ΰ΄€

𝑆Ω𝑙) … (π‘„π‘ˆ ΰ΄€ 𝑆Ω1)𝑁 To show that

  • π‘„π‘ˆπ΅π‘™ converges to 𝑁 w.h.p., it is

sufficient to show that the π‘„π‘ˆ ΰ΄€ 𝑆Ω operation makes matrices β€œsmaller” with high probability. Once

  • the error is small enough, we then take
  • ne final step to satisfy all conditions
  • simultaneously. For details, see [Rec11].
slide-40
SLIDE 40

Part VI: Open Problems

slide-41
SLIDE 41

Open Problems

  • For which tensors π‘ˆ can we show that SOS

gives exact tensor completion? We’ve shown it when π‘ˆ is orthogonal, but this can very likely be extended.

  • Important subproblem: When can we find 𝐡

such that 𝐡 𝑀𝑗 βŠ— π‘₯𝑗 = 𝑣𝑗 for all 𝑗 and |𝐡 𝑣, 𝑀, π‘₯ | ≀ 1 for all unit 𝑣, 𝑀, π‘₯?

  • Barak and Moitra [BM16] show that SOS solves

the approximate tensor completion problem in a somewhat broader setting with a different

  • analysis. Can these analyses assist each other?
slide-42
SLIDE 42

References

  • [BM16] B. Barak and A. Moitra, Noisy tensor completion via the sum-of-squares

hierarchy, COLT, JMLR Workshop and Conference Proceedings, vol. 49, JMLR.org p. 417–445, 2016

  • [PS17] A. Potechin and D. Steurer. Exact tensor completion with sum-of-squares.

COLT 2017

  • [Rec11] B. Recht. A Simpler Approach to Matrix Completion. JMLR Volume 12, p.

3413-3430. 2011

slide-43
SLIDE 43

Appendix: 𝜈0 and 𝜈 Definitions

slide-44
SLIDE 44

𝜈0 and 𝜈 Definitions

Definition:

  • 𝜈0 =

π‘œ 𝑠 β‹… max{max𝑏 𝑄𝑉𝑓𝑏 2 , max 𝑐

π‘„π‘Šπ‘“π‘

2}

Definition:

  • 𝜈 = n β‹… max{max𝑗,𝑏 𝑣𝑗𝑏

2 , max π‘˜,𝑐 π‘€π‘˜π‘ 2 , max 𝑙,𝑑 π‘₯𝑙𝑑 2 }