1 E. Boros, G.Felici Boolean Seminar, Liblice, April 2013 - - PowerPoint PPT Presentation
E. Boros, G.Felici Boolean Seminar, Liblice, April 2013 Some Results on Threshold Separability of Boolean Functions Endre Boros Giovanni Felici RUTCOR Istituto di Analisi dei Sistemi ed Informatica Rutgers University Consiglio Nazionale
disjoint sets
points in multidimensional space, find a set of meaningful rules that is able to separate records of different sets with high precision
Binarization of real valued data Each real valued dimension is mapped into a set of intervals, to achieve:
Binarizion is very important: is at the top of the analysis hierarchy For the i-th coordinate, define a set of cutponts
Ti = {ti1,ti2, …, ti,ki}, tij < ti,j+1
Define binary zij, j = 1,…, ki associated with Ti where
zij = 1 if tij-1 xi tij 0 otherwise
A binarization induces a set of boxes B in the space of S, each defined by the interserction of the subspaces parallel to the axes that intersect the cutpoints. With proper assumptions and bounding on the cutpoint sets, each box is closed and the set
Definitions
cutpoint set T.
either contain positive or negative points From P N = , we know that there always exists a separable binarization
into two non intersecting subsets
a simple way to obtain a separable binarization is to insert a cutpoint between each pair of consecutive points of different class Trivial conditions on set S guarantee the existance of a separable T-binarization and thus of a PDBF on Z
Linear Threshold Boolean Functions
f:{0,1}n → {-1,1} expressed as f(x) = sgn(c0 +c1x1+…+cnxn)
for S if and only if f(z) = 1 for z in P and f(z) = -1 for z in N.
We are interested in the conditions for Z to admit a LTF extension
existance of LTF-extensions are preserved
1. conditions for the existence of a Separating Linear Threshold Function 2. combinatorial necessary and sufficient condition for the existence of such function when points belong to the plane 3. Minimal forbidden structures for the existance of separating LTFs 4. insights for the problem in larger dimensions that can be practically used in distcretization / binarization algorithms to find good LTF.
z21=1 z11=1
21 11 21 21 11 11 11 21 21 21 11 11 01 11 21 21 11 11 10 21 21 11 11 00
21 11
The Forbidden Cross (FC)
Any set composed of 3 boxes is LTF- extendable (trivial) Equivalent representations: Positive Negative
A Minor is composed by the 4 “corner” boxes of a larger subset of B
admits a LTF representation if B does not contain any Forbidden Cross (FC) as a minor We like minimal structures that break the LTF representability of Z; this would be very useful to determine algorithmically the right T for S
The spanning operation: each time an empty box is the corner of a potential FC, we assign that box to the only class (color) that avoids the FC
A set of boxes is LTF-maximal when no spanning can be applied
the sequence of spanning
existence of a LTF-extension Claim 1: Z admits no LTF-extensions iif its LTF-maximal box sets contains a FC
Given a box set B, construct the following graph G=(V,E):
in the same column or row In the example: an alternating cycle of length 4 corresponds to a FC. What happens to longer alt-cycles ?
Spanning and alternating cycles Any alt-cycle of length 2n in G can generate an alt-cycle of length 2(n-1) by a spanning
1 cycle of length 6 + 1 cycle of length 4 + another cycle of length 4 + another cycle of length 4…
Claim 2: if G contains an alternating cycle, then no LTF-extension is possible
Important questions: is spanning allowed ? What are its relation with LTF-extensions? If spanning preserves the LTF extendability, then:
Claim 1: Z admits no LTF-extension iif its LTF-maximal box sets contains a FC Claim 2: if G contains an alternating cycle, then no LTF-extension is possible
are theorems… To make sure, we need some linear programming.
LTF-separabilty and Linear Programming (in the plane, for the time being) Given S R2, T, and Z:
2 1 1 2 1 1
i ij ij i ij ij i ij ij i ij ij
2 1 2 1 2 1
k k k j j ij k j j ij k j j ij k j j ij
x and y are the weights that are associated with the cutpoints in the two coordinates to determine the LTF. So far, we are interested only in feasibility
Remember that we may use
LTF Existence Conditions
2 1 ,
k k P P
(u,v) are the dual variables associated with constraints on P and N boxes
t t
In compact form And the dual is:
t t
that has empty feasible region if this has solution:
(1) (3) (2) i. Z admits a LTF representation (1) has solution ii. (1) has solution (2) has solution iii. If (3) has solution, (2) has no solution iv. A feasible solution of (3) represents a forbidden structure for LTF-extension
Conditions that define a Forbidden Minor
Consider (3):
t t
In detail:
| | 1 | | 1
N i i ij P i i ij
i. In a “slice” of the plane the sum of the weights of positive boxes is equal to the sum of the weights of negative boxes. ii. At least 2 v and 2 u must be > 0 iii. Must be valid for at least one horizontal and one vertical slice iv. In any slice, if a positive box is present, also a negative one must appear, and viceversa v. When this happens, no LTF extension is possible
16 Conditions for the existence of a Forbidden Minor The FC satisfies (3) and thus it is a forbidden minor (we knew that, already…) ) , ( ' ' min v u v e u e vB uB vA uA v u
t t
Any alternating cycle in the graph G previously defined satisfies (3) This proves that:
Claim 1: Z admits no LTF-extension iif its LTF-maximal box sets contains a FC Claim 2: if G contains an alternating cycle, then no LTF-extension is possible
LTF-test: span all boxes, check for FC presence
Assign 0 weight to boxes not in the FC, and same weight to the 4 boxes in the cross Assign 0 weight to boxes not in the cycle, and same weight to the boxes in the cycle
In higher Dimensions
For example, in R3 we
structure supported by 6 boxes in 3x3x3 Any permutation
a Forbidden Minor
structure in (n+1)-dimension
at least one positive and one negative box solves problem (3) and forbids LTF-extension
t t
Another plane Another plane Another plane Another plane Another plane Another plane
Alternating paths in graph for higher dimensions (1)
t t
that lie in the same plane (subspace)
2n solves (3)
requires n positive and n negative boxes… we call it Basic Minor in 3d (BM3)
that well known 4FC2?
Alternating paths in graph for higher dimensions (2)
t t
4 boxes in dimension 3 (projected onto 2x2x2)
A new type of FC in 3D, made of 4 boxes. We call it 4FC3 Obviusly, it would satisfy (3)
Unbalanced structures ?
Now we find 4FC3 We call this the 5-forbidden cross (5FC3). It can be proven to be the only minimal forbidden structure of 5 boxes in the space It is obtained as an alternating cycle after spanning w.r.t. to 4FC2
….but, there is no cycle! Apply spanning w.r.t. to 4FC2 If we assing to one box double weight, it solves (3)
Other structures in 3 dimensions:
Other structures in 4 dimensions: Claim: all the Forbidden Structures are alternating cycles on the proper G graph once the box system has been spanned according to FC, 4FC3, 5FC3, BM Claim: if no FC, 4FC3, 5FC3, BM is found once the box system has been spanned accordingly, then there exists a LTF extension. No counterexamples so far…
Objective: given S, find a “nice” LTF representation, if any. We can trivially verify its existance by solving the separation LP (1). What if (1) has no solution? Find
Our knowledge of LTF forbidden minors may help. Following, some algorithmic ideas that are being considered