-Testing Sofya Raskhodnikova Penn State University, visiting - - PowerPoint PPT Presentation

β–Ά
testing
SMART_READER_LITE
LIVE PREVIEW

-Testing Sofya Raskhodnikova Penn State University, visiting - - PowerPoint PPT Presentation

Aug 31, 2022 383 likes β€’682 views

-Testing Sofya Raskhodnikova Penn State University, visiting Boston University and Harvard Joint work with Piotr Berman (Penn State), Grigory Yaroslavtsev (Penn State Brown) 1 Property Testing Models Tolerant Property Tester

slide-1
SLIDE 1

1

π‘€π‘ž-Testing

Joint work with Piotr Berman (Penn State), Grigory Yaroslavtsev (Penn State β†’ Brown)

Sofya Raskhodnikova

Penn State University, visiting Boston University and Harvard

slide-2
SLIDE 2

2

Tolerant Property Tester [Parnas Ron Rubinfeld 06]

Far from YES

YES

Reject with probability 2/3 Don’t care Accept with probability β‰₯ πŸ‘/πŸ’

ο‚³

Property Testing Models

Equivalent to tolerant testing: estimating distance to the property. Two objects are at distance 𝜁 = they differ in an 𝜁 fraction of places

Property Tester [Rubinfeld Sudan 96,

Goldreich Goldwasser Ron 98]

Close to YES

Far from YES

YES

Reject with probability 2/3 Don’t care Accept with probability β‰₯ πŸ‘/πŸ’

ο‚³

𝜁 𝜁1 𝜁2

slide-3
SLIDE 3

Why Hamming Distance?

  • Nice probabilistic interpretation

– probability that two functions differ on a random point in the domain

  • Natural measure for

– algebraic properties (linearity, low degree) – properties of graphs and other combinatorial objects

  • Motivated by applications to probabilistically checkable

proofs (PCPs)

  • It is equivalent to other natural distances for

– properties of Boolean functions

3

slide-4
SLIDE 4

Which stocks grew steadily?

Data from http://finance.google.com

slide-5
SLIDE 5

π‘€π‘ž-Testing

for properties of real-valued data

slide-6
SLIDE 6

Use π‘€π‘ž-metrics to Measure Distances

  • Functions 𝑔, 𝑕: 𝐸 β†’ 0,1 over (finite) domain 𝐸
  • For π‘ž β‰₯ 1

π‘€π‘ž 𝑔, 𝑕 = 𝑔 βˆ’ 𝑕

π‘ž = π‘¦βˆˆπΈ

𝑔 𝑦 βˆ’ 𝑕 𝑦

π‘ž 1/π‘ž

𝑀0 𝑔, 𝑕 = 𝑔 βˆ’ 𝑕

0 =

𝑦 ∈ 𝐸: 𝑔 𝑦 β‰  𝑕 𝑦

  • π‘’π‘ž 𝑔, 𝑕 =

π’ˆ βˆ’π‘• π‘ž 1 𝒒

6

slide-7
SLIDE 7

7

Tolerant Property Tester

Far from YES

YES

Reject with probability 2/3 Don’t care Accept with probability β‰₯ πŸ‘/πŸ’

ο‚³

𝑴𝒒-Testing and Tolerant 𝑴𝒒-Testing

Property Tester

Close to YES

Far from YES

YES

Reject with probability 2/3 Don’t care Accept with probability β‰₯ πŸ‘/πŸ’

ο‚³

𝜁 𝜁1 𝜁2

Functions 𝑔, 𝑕: 𝐸 β†’ [0,1] are at distance 𝜁 if π‘’π‘ž =

π‘”βˆ’π‘• π‘ž 𝟐 π‘ž

= 𝜁.

slide-8
SLIDE 8

New π‘€π‘ž-Testing Model for Real-Valued Data

  • Generalizes standard 𝑀0-testing
  • For π‘ž > 0 still have a nice probabilistic interpretation:

distance π‘’π‘ž 𝑔, 𝑕 = 𝐅 π’ˆ βˆ’ 𝒉 𝒒

1/π‘ž

  • Compatible with existing PAC-style learning models

(preprocessing for model selection)

  • For Boolean functions, 𝑒0 𝑔, 𝑕 = π‘’π‘ž 𝑔, 𝑕 π‘ž.

8

slide-9
SLIDE 9

Our Contributions

  • 1. Relationships between π‘€π‘ž-testing models
  • 2. Algorithms

– π‘€π‘ž-testers for π‘ž β‰₯ 1

  • monotonicity, Lipschitz, convexity

– Tolerant π‘€π‘ž-tester for π‘ž β‰₯ 1

  • monotonicity in 1D (aka sortedness)

Our π‘€π‘ž-testers beat lower bounds for 𝑀0-testers Simple algorithms backed up by involved analysis Uniformly sampled (or easy to sample) data suffices

  • 3. Nearly tight lower bounds

9

slide-10
SLIDE 10

Implications for π‘΄πŸ-Testing

Some techniques/observations/results carry over to 𝑀0-testing

– Improvement on Levin’s work investment strategy Gives improvements in run time of testers for

  • Connectivity of bounded-degree graphs [Goldreich Ron 02]
  • Properties of images [R 03]
  • Multiple-input problems [Goldreich 13]

– First example of monotonicity testing problem where adaptivity helps – Improvements to 𝑀0-testers for Boolean functions

10

slide-11
SLIDE 11

Relationships between π‘€π‘ž-Testing Models

slide-12
SLIDE 12

Relationships Between π‘€π‘ž-Testing Models

𝐷𝒒(𝑸,𝜻) = complexity of 𝑀𝒒-testing property 𝑸 with distance parameter 𝜻

  • e.g., query or time complexity
  • for general or restricted (e.g., nonadaptive) tests

For all properties 𝑸

  • π‘€πŸ-testing is no harder than Hamming testing

𝐷𝟐(𝑸,𝜻) ≀ 𝐷𝟏(𝑸,𝜻)

  • 𝑀𝒒-testing for 𝒒 > 1 is close in complexity to π‘€πŸ-testing

𝐷𝟐(𝑸,𝜻) ≀ 𝐷𝒒(𝑸,𝜻) ≀ 𝐷𝟐(𝑸,πœ»π’’)

12

slide-13
SLIDE 13

Relationships Between π‘€π‘ž-Testing Models

𝐷𝒒(𝑸,𝜻) = complexity of 𝑀𝒒-testing property 𝑸 with distance parameter 𝜻

  • e.g., query or time complexity
  • for general or restricted (e.g., nonadaptive) tests

For properties of Boolean functions π’ˆ: 𝐸 β†’ 0,1

  • π‘€πŸ-testing is equivalent to Hamming testing

𝐷𝟐(𝑸,𝜻) = 𝐷𝟏(𝑸,Ξ΅)

  • 𝑀𝒒-testing for 𝒒 > 1 is equivalent to π‘€πŸ-testing

with appropriate distance parameter

𝐷𝒒(𝑸,𝜻) = 𝐷𝟐(𝑸,πœ»π’’)

13

slide-14
SLIDE 14

Relationships: Tolerant π‘€π‘ž-Testing Models

𝐷𝒒(𝑸,𝜻𝟐, πœ»πŸ‘) = complexity of tolerant 𝑀𝒒-testing property 𝑸 with distance parameters 𝜁1, 𝜁2

  • E.g., query or time complexity
  • for general or restricted (e.g., nonadaptive) tests

For all properties 𝑸

  • No obvious relationship between tolerant π‘€πŸ-testing

and tolerant Hamming testing

  • 𝑀𝒒-testing for 𝒒 > 1 is close in complexity to π‘€πŸ-testing

𝐷𝟐(𝑸,𝜻𝟐

𝒒, Ξ΅2) ≀ 𝐷𝒒(𝑸,𝜻𝟐, πœ»πŸ‘) ≀ 𝐷𝟐(𝑸,𝜻𝟐, πœ»πŸ‘ 𝒒)

14

slide-15
SLIDE 15

Relationships: Tolerant π‘€π‘ž-Testing Models

𝐷𝒒(𝑸,𝜻𝟐, πœ»πŸ‘) = complexity of tolerant 𝑀𝒒-testing property 𝑸 with distance parameters 𝜁1, 𝜁2

  • E.g., query or time complexity
  • for general or restricted (e.g., nonadaptive) tests

For properties of Boolean functions π’ˆ: 𝐸 β†’ 0,1

  • π‘€πŸ-testing is equivalent to Hamming testing

𝐷𝟐(𝑸,𝜻𝟐, πœ»πŸ‘) = 𝐷𝟏(𝑸,𝜻𝟐, πœ»πŸ‘)

  • 𝑀𝒒-testing for 𝒒 > 1 is equivalent to π‘€πŸ-testing

with appropriate distance parameters d

𝐷𝒒(𝑸,𝜻𝟐, πœ»πŸ‘) = 𝐷𝟐(𝑸,𝜻𝟐

𝒒, πœ»πŸ‘ 𝒒)

15

slide-16
SLIDE 16

𝑃𝑣𝑠 π‘†π‘“π‘‘π‘£π‘šπ‘’π‘‘

Property: Monotonicity

slide-17
SLIDE 17

Monotonicity

  • Domain D=[π‘œ]𝑒 (vertices of 𝑒-dim hypercube)
  • A function 𝑔: 𝐸 β†’ R is monotone

if increasing a coordinate of 𝑦 does not decrease 𝑔 𝑦 .

  • Special case 𝑒 = 1

𝑔: [π‘œ] β†’ R is monotone ⇔ 𝑔 1 , … 𝑔(π‘œ) is sorted. One of the most studied properties in property testing

[ErgΓΌn Kannan Kumar Rubinfeld Viswanathan , Goldreich Goldwasser Lehman Ron, Dodis Goldreich Lehman R Ron Samorodnitsky, Batu Rubinfeld White, Fischer Lehman Newman R Rubinfeld Samorodnitsky, Fischer, Halevy Kushilevitz, Bhattacharyya Grigorescu Jung R Woodruff, ..., Chakrabarty Seshadhri, Blais R Yaroslavtsev, Chakrabarty Dixit Jha Seshadhri]

17

(1,1,1) (𝑒, 𝑒, 𝑒)

slide-18
SLIDE 18

Monotonicity Testers: Running Time

18

𝑔 𝑀0 π‘€π‘ž π‘œ β†’ [0,1]

Θ log π‘œ 𝜻

[ErgΓΌn Kannan Kumar Rubinfeld Viswanathan 00, Fischer 04]

Θ 1 πœ»π‘ž

π‘œ 𝑒 β†’ [0,1]

Θ 𝑒 β‹… log π‘œ 𝜻

[Chakrabarty Seshadhri 13]

O

𝑒 πœ»π‘ž log 𝑒 πœ»π‘ž

Ξ©

1 πœ»π‘ž log 1 πœ»π‘ž for 𝑒 = 2

nonadaptive 1-sided error

slide-19
SLIDE 19

Monotonicity Testers: Running Time

19

𝑔 𝑀0 π‘€π‘ž π‘œ β†’ {0,1}

Θ 1 𝜻 Θ 1 πœ»π‘ž

π‘œ 𝑒 β†’ {0,1}

Θ 𝑒 𝜻 β‹… log3 𝑒 𝜻

[Dodis Goldreich Lehman R Samorodnitsky 99]

O

𝑒 πœ»π‘ž log 𝑒 πœ»π‘ž

Ξ©

1 πœ»π‘ž log 1 πœ»π‘ž for 𝑒 = 2

nonadaptive 1-sided error Θ

1 πœ»π‘ž

for constant 𝑒 adaptive 1-sided error

slide-20
SLIDE 20

20

𝑀1-Testing of Monotonicity

slide-21
SLIDE 21

Monotonicity: Reduction to Boolean Functions

Given π’ˆ: 𝐸 β†’ [0,1], a Boolean threshold function π’ˆ(𝒖): 𝐸 β†’ {0,1} π’ˆ(𝒖) 𝑦 = 1 if π’ˆ 𝑦 β‰₯ 𝑒 0 otherwise

  • Decomposition: 𝑔 𝑦 =

1 π’ˆ(𝒖) 𝑦 𝑒𝒖

  • M = class of monotone functions

Characterization Theorem

𝑀1 π’ˆ, 𝑁 = 0

1𝑀1 π’ˆ(𝒖), 𝑁 𝑒𝒖

21

1 π’ˆ(𝒖) 𝑦 𝒖 π’ˆ 𝑦

slide-22
SLIDE 22

Characterization Theorem: One Direction

  • βˆ€π‘’ ∈ 0,1 , let 𝑕𝑒=closest monotone (Boolean) function to π’ˆ(𝒖).
  • Let 𝒉 = 0

1𝑕𝑒𝑒𝒖. Then 𝒉 is monotone, since 𝑕𝑒 are monotone.

22

𝑀1 π’ˆ, 𝑁 ≀ 𝑔 βˆ’ 𝑕 1 =

1π’ˆ(𝒖)𝑒𝒖 βˆ’ 0 1𝑕𝑒𝑒𝒖 1

=

1(π’ˆ(𝒖)βˆ’π‘•π‘’)𝑒𝒖 1

≀ 0

1 π’ˆ(𝒖) βˆ’ 𝑕𝑒 1𝑒𝒖

= 0

1𝑀1 π’ˆ(𝒖), 𝑁 𝑒𝒖

Because 𝒉 is monotone Decomposition & definition of 𝒉 Triangle inequality Definition of 𝑕𝑒

𝑀1 π’ˆ, 𝑁 ≀ 0

1𝑀1 π’ˆ(𝒖), 𝑁 𝑒𝒖

slide-23
SLIDE 23

Monotonicity: Using Characterization Theorem

We use Characterization Theorem to get monotonicity 𝑀1-testers and tolerant testers from standard property testers for Boolean functions.

23

Characterization Theorem

𝑒1 π’ˆ, 𝑁 = 0

1𝑒1 π’ˆ(𝒖), 𝑁 𝑒𝒖

slide-24
SLIDE 24

𝑀1-Testers from Testers for Boolean Ranges

A nonadaptive, 1-sided error 𝑀0-test for monotonicity of 𝑔: 𝐸 β†’ {0,1} is also an 𝑀1-test for monotonicity of 𝑔: 𝐸 β†’ [0,1]. Proof:

  • A violation (𝑦, 𝑧):
  • A nonadaptive, 1-sided error test queries a random set 𝑅 βŠ† 𝐸

and rejects iff 𝑅 contains a violation.

  • If 𝑔: 𝐸 β†’ [0,1] is monotone, 𝑅 will not contain a violation.
  • If 𝑒1 𝑔, 𝑁 β‰₯ 𝜁 then βˆƒπ’–βˆ—: 𝑒0 π’ˆ(π’–βˆ—), 𝑁 β‰₯ 𝜻
  • W.p. β‰₯ 2/3, set 𝑅 contains a violation (𝑦, 𝑧) for π’ˆ(π’–βˆ—)

π’ˆ(π’–βˆ—) 𝑦 = 1, π’ˆ(π’–βˆ—) 𝑧 = 0 ⇓ π’ˆ 𝑦 > π’ˆ 𝑧

24

π’ˆ(π’š) π’ˆ(𝒛) >

slide-25
SLIDE 25

Monotonicity Testers: Running Time

25

𝑔 𝑀0 π‘€π‘ž π‘œ β†’ [0,1]

Θ log π‘œ 𝜻

[ErgΓΌn Kannan Kumar Rubinfeld Viswanathan 00, Fischer 04]

Θ 1 πœ»π‘ž

π‘œ 𝑒 β†’ [0,1]

Θ 𝑒 β‹… log π‘œ 𝜻

[Chakrabarty Seshadhri 13]

O

𝑒 πœ»π‘ž log 𝑒 πœ»π‘ž

Ξ©

1 πœ»π‘ž log 1 πœ»π‘ž for 𝑒 = 2

nonadaptive 1-sided error

slide-26
SLIDE 26

Distance Approximation and Tolerant Testing

26

𝑔 𝑀0 𝑀1 π‘œ β†’ [0,1]

polylog π‘œ β‹… 𝟐 𝜺

𝑷 𝟐/𝜺 [Saks Seshadhri 10]

Θ 𝟐 πœΊπŸ‘ Approximating π‘΄πŸ-distance to monotonicity ±𝜺 𝒙. 𝒒. β‰₯ πŸ‘/πŸ’

  • Time complexity of tolerant 𝑀1-testing for monotonicity is

O πœ»πŸ‘ (πœ»πŸ‘ βˆ’ 𝜻𝟐)πŸ‘ .

slide-27
SLIDE 27

𝑀1-Testers for Other Properties

Via combinatorial characterization of 𝑀1-distance to the property

  • Lipschitz property π’ˆ: 𝒐 𝒆 β†’ [0,1]:

Θ

𝒆 πœ— (tight)

Via (implicit) proper learning: approximate in 𝑀1 up to error 𝝑, test approximation on a random 𝑃(1/πœ—)-sample

  • Convexity π’ˆ: 𝒐 𝒆 β†’ [0,1]:

O π‘βˆ’π’†

2 +

1 𝝑 (tight for 𝒆 ≀ 2)

  • Submodularity π’ˆ: 0,1 𝒆 β†’ 0,1

2

𝑃 1

𝝑 + π‘žπ‘π‘šπ‘§

1 𝝑 log 𝒆 [Feldman Vondrak 13]

slide-28
SLIDE 28

Open Problems

  • Our 𝑀1-tester for monotonicity is nonadaptive, but we

show that adaptivity helps for Boolean range. Is there a better adaptive tester?

  • All our algorithms for π‘€π‘ž-testing for π‘ž β‰₯ 1 were
  • btained directly from 𝑀1-testers.

Can one design better algorithms by working directly with π‘€π‘ž-distances?

  • We designed tolerant tester only for monotonicity

(d=1,2). Tolerant testers for higher dimensions? Other properties?

28