Roadmap for Section 12.2. Registry Fundamentals Registry Structure - - PDF document

1

Windows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas Polze

Unit OS12: Scripting

12.2. The Registry

3

Roadmap for Section 12.2.

Registry Fundamentals Registry Structure Registry Limits Monitoring Registry Activity with Regmon

2

4

Registry Fundamentals

Repository for system/user configuration information

Contains information NT requires to boot & configure System-wide application settings (e.g. install directory) Per-user settings (e.g. fonts, window placement, etc.)

Called Configuration Manager (CM) in NTOSKRNL Network accessible (via Remote Registry service)

Provides limited remote configuration, but not management

Changing some registry entries causes action, but only if something watching for changes

WMI provides remote management (described in next section)

Also is a window into in-memory (volatile) system information

Hardware description (built during boot process) Performance data

5

Registry Interface to Performance Counters

Custom application A Custom application B Performance tool

Windows Management Instrumentation High-performance provider interface Registry DLL provider

Pdh.dll Advapi32.dll

RegQueryValueEx

PerfLib System performance DLL Performance extension DLL High-performance data provider

• bject

Performance- monitoring applications Programming Interfaces

3

6

Viewing and Changing the Registry

Should never have to do this!

Most common Registry parameters can be adjusted using a graphical NT utility But, some advanced tuning and configuration options require modifying and/or adding registry data

Registry Editor tools

Windows NT4/2000 had 2 tools (Regedt32.exe and Regedit.exe)

REGEDT32.EXE

Interprets NT-specific data types, NT permission lists

REGEDIT.EXE

Doesn’t understand NT-specific data types or security But has more thorough find capability

XP/2003: single combined Regedit.exe does everything No shortcuts for these tools

7

Command Line Registry Tools

Reg.exe (built into XP/2003; also in 2000 Support Tools)

Can compare keys, load/unload hives, add/remove/query keys, and backup/restore (useful since NT Backup can only backup local registry hives)

Regini.exe (built in to XP/2003; also in 2000 Resource Kit) - make registry changes via a script (.INI file) Regdmp (2000 Resource Kit) – dump registry in format that Regini reads Scanreg.exe (2000 Resource Kit) - flexible registry search tool

4

8

Registry Usage

When is the Registry read?

During boot (to determine which drivers, configure the system) During login (to set user preferences, such as drive letter mappings, screen saver, wallpaper, etc.) When you start applications (to read app-specific settings and per- user settings)

When is the Registry modified?

When you install applications or drivers When you change system, user or application settings

It should not be polled for changes

Interesting to check your systems for “quietness” of Registry Should use Registry change notification Windows functions

Thread sleeps until a change is made to the part of the Registry they are interested in

9

Registry Structure

Registry editors display Registry as an Explorer-like tree view

Lab: run Regedit and examine

The Registry is structured like a file system:

Keys are like directories Values are like files Values have data types – like a file type Data is like the contents of a file

The most common value types (there are 12 total):

REG_DWORD - 32-bit integer REG_SZ - string REG_MULTISZ – array of strings REG_BINARY – array of bytes to represent arbitrary data

Registry keys have full NT security

Allows fine grained control as to who has what access

5

11

Logical Registry Structure

Five visible “root” keys organize the data

Really only two “roots”: HKLM and HKU

• 1. HKEY_CLASSES_ROOT (HKCR)

Link to HKLM\Software\Classes - file extension associations, COM class registrations

HKEY_LOCAL_MACHINE (HKLM)

System-global static and dynamic configuration

HKEY_USERS (HKU)

Root for the loaded (logged on) user profiles

HKEY_CURRENT_USER (HKCU)

Link to profile of the user accessing this key

HKEY_CURRENT_CONFIG (HKCC)

Link to subset of HKEY_LOCAL_MACHINE - has hardware profile information

Hidden virtual key for performance data

HKEY_PERFORMANCE_DATA

12

HKEY_LOCAL_MACHINE

Five main subkeys for local machine information

HKEY_LOCAL_MACHINE\Hardware

Hardware configuration data, resource usage, etc. Completely volatile (not saved across boots)

HKEY_LOCAL_MACHINE\SAM

Local account & groups database (for NT4 domain controllers, served as the domain account database)

HKEY_LOCAL_MACHINE\Security

System-wide security policies (for NT4 domain controllers, served as the domain security policy database)

HKEY_LOCAL_MACHINE\Software

Per-machine software data not critical for booting

HKEY_LOCAL_MACHINE\System

Controls booting and running the system (config.sys functionality) Available during bootstrap

6

13

HKEY_USERS

HKEY_USERS is root for loaded user profile Registry data

This is stored in \Documents and Settings\%USERNAME%\ntuser.dat

In NT 4 it was stored in \Winnt\Profiles\%USERNAME%

Note this is just a small part of your user profile, which includes your desktop, start menu, My Documents, etc.

A user profile is “loaded”:

When a user logs on interactively When a service process configured to run under a specific user account starts RUNAS (unless /NOPROFILE specified)

14

User Profiles

HKEY_USERS\.Default is NOT the default Registry profile

It is the profile for the System account

E.g. Change logon desktop settings are here because Winlogon runs under System

The default user Registry profile is in:

\Documents and Settings\Default User\Ntuser.dat

The starting user profile for users logging without a profile

This does not show up in the Registry because it is not loaded

Note: roaming profiles are left on a system even after you logoff!

Anyone with local administrative privileges can view your documents and settings and Registry information Can override with group policies

7

15

Registry Physical Structure

Registry consists of a number of files called “hives”

A hive is a piece of registry database stored in its own file System-wide configuration information is in %systemroot%\system32\config User profile (HKEY_CURRENT_USER\...) in \Documents and Settings\…\ntuser.dat

User profile is more than just the registry settings Entire directory under \Documents and Settings (e.g. start menu, My documents, temporary files, etc)

Profile for new users is in \Documents and Settings\Default User

NTLDR loads System hive, which points to other hives

Registry points to loaded hives at: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\hivelist

Can defrag registry hives with Pagedfrg from Sysinternals

16

Backup Hives

.sav hive files in \Windows\System32\Config are the template hive files stored by Windows Setup \Windows\Repair has copies of the hives that were made after Setup completed

Windows Backup updates them when it backs up “System State”

System Restore restore points (described in the startup/shutdown section) also store copies of registry hives

8

17

Registry Limits

SYSTEM hive was limited to 12MB in Windows 2000

Now limited to 200 MB or # of RAM, whichever is lower Limit is due to fact that entire SYSTEM hive is loaded into memory by NTLDR (discussed in Startup and Shutdown section)

Total loaded registry hive data was limited to 376MB in Windows 2000

This was because registry hives were read into paged pool when loaded

Explains why there was a system registry quota (see Control Panel- >System->Performance Tab - click on Virtual Memory “Change” button)

Limited number of terminal server users

XP/2003: No limit to loaded registry hive data

Hives are accessed as memory mapped files

Views are mapped as necessary

Registry no longer in paged pool System registry quota is gone

18

Registry Reliability & Recovery

Registry mechanism guarantees atomicity

Uses transaction logging to ensure structural integrity One .LOG file per registry hive Automatically used to recover corrupt registry hives other than SYSTEM

SYSTEM hive corruption recovery:

Prior to Windows XP the system keeps a backup version of System called System.Alt

NtLdr uses System.Alt of System is corrupt

On Windows XP, Ntldr knows how to apply the registry log files to the System hive in order to repair it in cases where the system crashed during updates

On Windows Server 2003 the configuration manager “self-heals” by amputating parts of the registry that are corrupt

9

19

Vista Transactions

The Vista Registry supports transactions

Allows application file system and Registry changes to be undone Changes committed only when associated transaction commits Relies on Distributed Transaction Coordinator (DTC) on Vista Server to enable multi-node transactions

Transaction API is with NTFS (TxF) and the Registry

Documented in Vista Beta 1 SDK CreateTransaction, SetCurrentTransaction, CommitTransaction, RollbackTransaction

20

Configuration Problems

Missing, corrupted or overly-secure Registry settings often lead to application crashes and errors Some applications don’t completely remove registry data at uninstall Regmon may yield the answer…

10

21

How Regmon Works

Regmon uses a driver to intercept Registry operations Up until now Regmon has relied on system call “hooking” to intercept Registry accesses

Hooking isn’t supported by the kernel As of XP the system call table is write- protected by default if a system has <256 MB, requiring a trick

Server 2003 introduces a Registry callback mechanism

Driver can see and modify Registry behavior Latest version of Regmon comes with two drivers: one for Server 2003 and one for previous versions Defined in a DDK header file and used by antivirus products Application Regmon Driver Regmon GUI Registry Subsystem

22

How Regmon Works

Regmon catches all Registry calls that pass through the system service dispatcher

Does not see direct calls to the native Registry API from kernel-mode code

Regmon has to dynamically obtain system call numbers to index into the system service dispatch table

System call numbers are assigned by scripts in the build process and can change between service packs All system call functions start with code like this:

MOVE SYSCALL # -> REGISTER EXECUTE SYSTEM CALL

Regmon reads the system call numbers out of the functions

11

23

Regmon Output

Request: OpenKey, CreateKey, SetValue, QueryValue, CloseKey Path

HKCU=HKEY_CURRENT_USER (per-user settings) HKLM=HKEY_LOCAL_MACHINE (system wide settings)

Result – return code from Registry operation Other – extended information or results

24

Controlling Regmon

Start/stop logging (Control/E) Clear display (Control/X) Regedit jump (opens Registry Editor and jumps directly to key)

Double clicking on a line does this

Filtering/Highlighting Find Save to log file

12

25

Regmon Filtering

Filtering options:

Process name or registry path (or partial name) Success/failure, read/write

26

Troubleshooting with Regmon

Two basic techniques:

Go to end of log and look backwards to where problem occurred or is evident and focused on the last things done Compare a good log with a bad log

Often comparing the Registry activity of a failing process with one that works may point to the problem

Have to first massage log file to remove data that differs run to run

Delete first 3 columns (they are always different: line #, time, process id)

Easy to do with Excel by deleting columns

Then compare with FC (built in tool) or Windiff (Resource Kit)

13

27

Troubleshooting with Regmon

Sometimes examining the values read and/or written (in Other column) may show up corrupt or wrong configuration data Sometimes queries to what is not there is more interesting than what is there Identify missing Registry keys

Search for status “NOTFOUND”

For permission problems, Search for status “ACCESS DENIED” Note: BUFROVRFLOW is normal

Applications query a value’s size by passing a 0-length buffer They are returned the length of the data and the overflow error and then they allocate a correctly-sized buffer and repeat the operation

28

Troubleshooting with Regmon

If you suspect registry data is causing problems, rename the key and re-run the application

Most applications re-create user settings when run In this way, the data won’t be seen by the application

Can always rename the key back

14

29

Regmon Example: Misleading IE Error Message

Internet Explorer failed to start with this error:

30

Solution: Misleading IE Error Message

Looked backwards from end of Regmon log Saw query of Completed value in Internet Connection Wizard key

Value read was 0 Value was 1 on other systems

Solution: Set value to 1 and problem went away

15

31

Example Problem

Problem:

User somehow disabled all toolbars and menus in Word No way to open files, change settings etc.

Solution:

With Regmon, captured startup of Word Found location of user-specific settings for Word Deleted this Registry key Re-ran Word, which re-created user settings from scratch

32

Example Problem

Internet Explorer hung when started

Default internet connection was set, but wasn’t being dialed

Dialing the connection first manually and then running IE worked

16

33

Solution

Ran Regmon Looked backwards from end (at the point IE was hung)

Found references to ATT under a PhoneBook key Renamed ATT key and problem went away

Background information:

User had previously installed the AT&T Dialer program, but had uninstalled it and created dial up connection manually

Conclusion: registry junk was left from uninstall

Note that uninstalls don’t clean up stuff in currently not loaded user profiles!

34

Example: Misleading VBA Error

User got this error installing an application: Regmon showed permissions problem Solution: Edited permissions

17

35

Missing Settings

Sometimes queries to what is not there is more interesting than what is there Identify missing Registry keys Search for status “NOTFOUND” May reveal hidden capabilities

36

Example: Hidden Capabilities

User tried installing Compaq utility on non-Compaq hardware Regmon showed hidden key to override: Solution: created key and install succeeded

18

37

Logging Boot Registry Activity

Regmon has a “log boot” option

Installs its driver to logs Registry activity from boot time to \Windows\Regmon.log Stops logging when you run Regmon

Regmon loads as very first driver so it captures virtually everything

Remember that it does not see direct calls to the native Registry API from kernel-mode code

Lab: Enable boot logging and search resulting log for Winlogon debug Registry values

38

Logging Activity in Unprivileged Accounts or During Logon/Logoff

May need to capture Registry activity in an unprivileged account

e.g. An application may work when run as Administrator, but not as a normal user Regmon requires the DEBUG and LOAD_DRIVER privileges to execute

Or, may need to capture Registry activity during the logon or logoff process

E.g. An Explorer shell extension is crashing Explorer during logon

In these cases Regmon needs to remain running after logoff

Only way to have a process not be destroyed at log off is to have it run as a service or as a child of a service process

• 1. For a quick, one-time execution, run with “psexec -i –s -d”

command

• 2. Install and run Regmon as a service using Srvany tool in

Resource Kit Can configure to start at system boot When you logon Regmon will still be running

19

39

Further Reading

Mark E. Russinovich and David A. Solomon, Microsoft Windows Internals, 4th Edition, Microsoft Press, 2004.

Chapter 4 - Management Mechanisms; The Registry (from pp. 183)