Superfetch : everything you need to know about privacy Mathilde - PowerPoint PPT Presentation

Superfetch : everything you need to know about privacy Mathilde Venault & Baptiste David c0c0n India 2020

About t us Laval, France Mathilde VENAULT venault@et.esiea.fr Baptiste DAVID bdavid@et.esiea.fr

What is is it it ? Resource Monitor view • SysMain = preloaded memory + preloaded processes + scenarios

Conte text : the service ice SysMain ain • The main goal is to increase speed eed of of us user er experie ex erienc nce through: ➢ Optimizing boot of the os. ➢ Anlayzing software use & prelaunching programs the user might need next time. • Misus isuse of of langua nguage ge: « Superfetch » is only a part of SysMain, which is the name of the whole service. It is often called Superfetch because on older Windows versions, the service was called Superfetch. Sysmain properties

Conte text : : Sysmain’s hea eadq dquar arte ters C:\Windows\Prefetch directory • SysMain stores its files on C:\Windows\Prefetch . • This directory includes : ▪ « ReadyBoot » directory related to the Readyboost functionnalities. ▪ Files related to the service (with .db and .pf extension ), traces of Superfetch’s activity. ▪ A file named « Layout.ini » which is the key file to speed up the boot.

Optim imiz izing ing the e boot • The goal : find the quickes est way for the OS to boot. • The list represents the best order to load the given files in memory. • Begins with the kernel! C:\Windows\Prefetch\Layout.ini

Mechan hanism sm : memo mory ry pagin ing

Mechani hanism sm : pages es faults lts

Mechan hanism sm : proces cessi sing page e faults lts (1)

Mechan hanism sm : proces cessi sing page e faults lts (2)

Mechani hanism sm : process cessin ing page e faults lts (3)

Mechan hanism sm : reducin ducing memory ory operations tions • Superfetch aims at reducing the occurrence of page faults, which require times & operations from memory. • To this end Superfetch : ▪ Remembers page accesses. ▪ Logs pages faults. ▪ Maps to physical memory pages referenced whenever the relative program is launched.

Global archi chitectu tecture e

Agen ent t Context xt (AgCx) • Deals session information based on SID & Token User. • Watches for conte text chang nge : ▪ hibernation (long pause). ▪ standby (short pause). ▪ fast user switching (change of user session). • Takes a snapshot of the situation when this is about to change. Includes two types of deconnection: ▪ Classic Disconnect (quitting & logging). ▪ « Lazy Disconnect » (without quitting).

Agen ent t PfnDb Db (AgPd) • Page Frame Number (PFN) is an array representing each physical page state of memory on the system ( Active / Standby / Freed …). • Logs page faults encountered by each program. • Classifies responses : ▪ Is it a « private page » ? (committed page) ▪ Is the page from a background app ?

Files es compress ssion ion RtlCompressBuffer() prototype, msdn.com Page Frame Number (PFN) is an array representing each physical page state of memory on the ▪ system (Active / Standby / Freed … ).

Agen ent t PfnDb Db (AgPd) • Page Frame Number (PFN) is an array representing each physical page state of memory on the system ( Active / Standby / Freed …). • Logs page faults encountered by each program. • Classifies responses : ▪ Is it a « private page » ? (committed page) ▪ Is the page from a background app ?

Agen ent t AppLaunch unch (AgAl) & Robust st perfor forman mance ce (AgRp Rp) AgAl • Post processes data received from FileInfo to take future decisions. • Creates Markov chains to represent probabilities of patterns use. AgRp • Calculates performance of predictions. • Assures relevance of the databases : ▪ Checking how many times/since when the data has been used. ▪ Calculating a « pertinence threshold » depending on other scenario use.

Agen ent t Gl Global (AgGl Gl) ) & Agen ent t AppLaunc nch (AgAl) AgGl • Organizes « histories » (individual history, fault history, global history). • Defines phases per days (morning/ week days, weekends..). AgAl • Make predictions depending on the Markov chains established before.

Global archi chitectu tecture e

Types s of super perfet fetch ch tasks • Pf routines tines ▪ « Non stop » job. ▪ Processing traces (building & updating scenarios), predicting and pre launching, daily checks.. • Perio iodic dic save ▪ Each 3 days in average, but depends on the value to save. ▪ Saving databases, updating registry keys… • Idle le tasks ks ▪ Under special circumstances (cpu, disk & memory utilization + power supply). ▪ Updating optimal layout ; launching « defrag.exe – s – b »

What about the prefetch files?

Files es compress ssion ion RtlCompressBuffer() prototype, msdn.com • All the prefetch files, except AgAppLaunch.db, AgRobust.db, dynrespri.db & cadrespri.db are compressed. • The files are compressed within the function RtlCompressBuffer() from NtosKrnl.lib. • The compression format is the XPRES ESS_ S_HUF UFFMA MAN format.

Databas base files es : generalities alities • Traces of agent’s activity : way to build internal database. One agent has 1 or more « .db ». • • They are not always present on the prefetch directory. • Until now, their format was undocumented. C:\Windows\Prefetch directory

Database se files es : name mes • AgAppLauch.db • AgCx_%SIDofUser.db • AgGlobalFaultHistory.db • AgForegroundAppHistory.db • AgGlobalHistory.db AgGlUserActiveDays_%sid (?) • • Dynamicreservedpriority.db C:\Windows\Prefetch directory

Database se files es : compress ssed ed forma mat AgCx.db : original file (compressed)

Database se files es: : decompr mpres esse sed forma mat AgCx.db (decompressed)

Database se files es: : decompr mpres esse sed forma mat AgCx.db (decompressed)

Database se Parame meters ters Type Parameters are defined for a specific • FileType pe. 16 different types of FileType. • 2 main uses: • offset calculation on the file. ▪ database size on core’s . ▪ DatabaseParams in SysMain.sys

Database se Parame meters ters Parameters are defined for a specific • FileType pe. 16 different types of FileType. • 2 main uses: • offset calculation on the file. ▪ database size on core’s . ▪ DatabaseParams in SysMain.sys

Database se files es: : from regis gistr try AgCx.db (decompressed) • Another source of databases ?

Datab abase ase files es : from regi gist stry ry AgCx.db : original file (compressed) AgCx.db : original file (compressed)

Database se files es : from regis gistr try ApLaunch.db : original file (decompressed)

Scen enarios arios files es : constr structio tion

Scen enarios arios files es : gener neralit alities ies • File ending with .pf : trace of an application. A same application could have one or more scenario file, depending on the context of its execution. • Each scenario file name is : « NameoftheApp – Hash .pf ». • Information defined on the registry : SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Prefetcher - MaxPrefetchFiles : by default 256. - MaxPrefetchFileSize : by default 10485 760 bytes.

rios files les : Scena enari : names es • The name result from the full application path hashed with the following algorithm : • Note that this algorithm depends on your Windows version. The following elements might change : ▪ Initalization value of string_hashed. ▪ Adding a modulo operation. ▪ Adding a multiplier coefficient.

Scen enarios arios files es : head eader er forma mat Prefetch signature OS version File Size Exe Name Exe hash Number of paths Offset where registered paths list begins Last execution dates Count of execution Scenario file : VLC.EXE-5A3EF7FA.pf (decompressed)

Scen enarios arios files es : content tent Contains the full paths of file needed to avoid • pagefault. In other words three kinds of files : • Non stopped consulted files, such as dll, ▪ dependencies. Recent files such as personal files. ▪ Caches files, because they are non ▪ stopped consulted. Scenario file : VLC.EXE-5A3EF7FA.pf (decompressed)

Scen enarios arios files es : cont nten ent Contains the full paths of file needed to avoid • pagefault. In other words three kinds of files : • Non stopped consulted files, such as dll, ▪ dependencies. Recent files such as personnal files. ▪ Caches files, because they are non ▪ stopped consulted. Scenario file : VLC.EXE-5A3EF7FA.pf (decompressed)

The e cache he files es Superfetch references cache files es. • Cache is a memory management which • stores temporarily data to reduce access time to these data later, in the cache files. Windows Cache directory

What about t the e conten tent t of the e file e ? Extract of a cache file • Superfetch references caches files. • … and cache files can contains data in clea ear from files es.

Time to practice!