{"id":781,"date":"2012-02-15T01:42:00","date_gmt":"2012-02-15T01:42:00","guid":{"rendered":"http:\/\/web03.partners.extranet.chrisse.com\/wordpress\/?p=781"},"modified":"2023-06-06T17:09:38","modified_gmt":"2023-06-06T15:09:38","slug":"how-the-active-directory-data-store-really-works-inside-ntds-dit-part-2","status":"publish","type":"post","link":"https:\/\/blog.chrisse.se\/?p=781","title":{"rendered":"How the Active Directory \u2013 Data Store Really Works (Inside NTDS.dit) \u2013 Part 2"},"content":{"rendered":"\n

This is the second post in a series of articles that will describe what’s really inside NTDS.dit and how Active Directory works on the database layer, In an earlier post I explained the tables within NTDS.dit in detail as far as what they are used for, in which release of Active Directory (Windows Server) they were introduced in, as well any major changes being added in later versions: How the Active Directory \u2013 Data Store Really Works (Inside NTDS.dit) \u2013 Part 1<\/a><\/p>\n\n\n\n

This post will go into the details of the contents of the “datatable” also known as the object store \u2013 that contains all objects and phantoms [1.1] represented as rows (1 object\/phantom = 1 row in the table) from any instanced naming context (NC) held as either writable or read-only (until they are physically removed by the garbage-collector) by the Directory System Agent (DSA) hosting the database and where columns represent every [1:3] attribute present in the schema except linked attributes [1:2]<\/p>\n\n\n\n

[1.1]: phantoms are references to object’s hosted outside the given database (NTDS.DIT) and the given Directory System Agent (DSA) \u2013 (Except structural phantoms)
<\/em><\/span><\/p>\n\n\n\n

[1:2] Post-Windows Server 2003 the attribute “ntSecurityDescriptor” is stored in the “sd_table” rather than in the “datatable”
<\/em><\/span><\/p>\n\n\n\n

[1:3] Some columns doesn’t reflect attributes and are columns pre-defined in the NTDS.dit template database generated by Microsoft (those are needed for internal states to the DSA)
<\/em><\/span><\/p>\n\n\n\n

Maintain the hierarchy of an object tree within a flat object store
<\/strong><\/p>\n\n\n\n

The hierarchy in Active Directory is quite obvious to most of us at a simplified layer e.g. daily administrative task such as creating an Organizational Unit and creates several descendent\/child objects under neat it, some people may refer to some objects as leaf objects (object that usually don’t contain descendent\/child objects) such as object of the class “user” \u2013 However the fact is that any object within Active Directory has the possibility (technically) to contain one or more descendent\/child objects \u2013 this is controlled by schema constrains and more specifically the sum of the following attributes of a given object class and any inherited class (except for auxiliary classes) :<\/p>\n\n\n\n

Table 1: Possible Superiors
<\/span><\/p>\n\n\n\n

\n<\/colgroup>\n\n\n\n\n
\n

Attribute<\/span><\/p>\n<\/td>\n

\n

Description<\/span><\/p>\n<\/td>\n<\/tr>\n

possSuperiors<\/span><\/td>\nContains references to object classes that can host the given as a descendent\/child object.<\/span>\n

 <\/p>\n

possSuperios can be modified on both cat1 and cat2 schema class object’s after that they have been instantiated in the schema.<\/p>\n<\/td>\n<\/tr>\n

systemPossSuperiors<\/span><\/td>\nContains references to object classes that can host the given as a descendent\/child object.
<\/span>\n

 <\/p>\n

systemPossSuperiors can’t be modified from the outside, once being instanced after the initial creation of the class within the schema.<\/span><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n\n\n\n

Why it’s easy for all objects to host descendants\/child objects becomes more obvious when the hierarchy is explained at the DBLayer.<\/p>\n\n\n\n

The question remains with the details given above, if one row within the “datatable” represents an object\/phantom, how can the hierarchy be maintained? The below table “Table 2” represent columns in the “datatable” that are vital for representing\/building the hierarchy in the directory at the DBLayer.<\/p>\n\n\n\n

Table 2: datatable \u2013 Simplified for hierarchy representation 1
<\/span><\/p>\n\n\n\n

\n<\/colgroup>\n\n\n\n\n\n
\n

Name<\/span><\/p>\n<\/td>\n

\n

ESE Data Type<\/span><\/p>\n<\/td>\n

\n

ESE Column Options (grbit)<\/span><\/p>\n<\/td>\n

\n

Description<\/span><\/p>\n<\/td>\n<\/tr>\n

DNT_col<\/span><\/td>\nJET_coltypLong<\/span><\/a><\/td>\nJET_bitColumnFixed,
JET_bitColumnAutoincrement<\/span><\/a><\/td>\n		Every object\/phantom within the “datatable” contains a unique DNT value. <\/span>\n

 <\/p>\n

DNT is a short for distinguished name tag.<\/p>\n

ESE enforces uniqueness by declaring DNT to be an ESE auto-incrementing column (JET_bitColumnAutoincrement<\/a>.)<\/p>\n

DNT is the primary key of the “datatable”, so [2.1] objects are clustered in storage by DNT, and access to an object by DNT is more efficient than access via any other column\/attribute. Since new objects are created in ascending DNT order, the primary key organization does not slow down the creation of new objects.<\/span><\/p>\n<\/td>\n<\/tr>\n

PDNT_col<\/span><\/td>\nJET_coltypLong<\/span><\/a><\/td>\nJET_bitColumnFixed<\/span><\/a><\/td>\nThe PDNT column holds the DNT of the parent of an object [2.2].<\/span>\n

 <\/p>\n

PDNT is a short for parent distinguished name tag.<\/span><\/p>\n

The tree structure of objects is not represented by pointers from parent to child, as you might expect given how the tree is normally browsed, but by a pointer in each child object\/row to its parent<\/p>\n<\/td>\n<\/tr>\n

RDNTyp_col<\/span><\/td>\nJET_coltypLong<\/span><\/a><\/td>\nJET_bitColumnFixed<\/span><\/a><\/td>\nThe RDNTyp_col holds the attributeID to the attribute being used as RDN, typically: cn (Common-Name<\/a>), ou (Organizational-Unit<\/a>), dc (Domain-Component<\/a>), o (Organization<\/a>) [2:3]<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n\n\n\n

[2.1]: The maximum numbers of objects\/phantoms that ever can be created on a given DSA (Domain Controller) for its entire life time is 2 billion objects (147,483,393 (231 minus 255)). Note that this count against objects\/phantoms ever introduced to the local DSA as part of any naming context (NC) writable or partial ever hosted by the DSA. * If the DSA is promoted by using IFM (Install from Media it inheritance the count of already allocated DNTs from the former DSA) \u2013 When the maximum numbers of auto-increment values has been used (the limit mention above have been hit) the following error are returned at the DBLayer: JET_errOutOfAutoincrementValues -1076<\/a> from the outside we will notice: “Error: Add: Operations Error. <1> Server error: 000020EF: SvcErr: DSID-0208044C, problem 5012 (DIR_ERROR), data -1076.” Read more about Active Directory Limits: http:\/\/technet.microsoft.com\/en-us\/library\/active-directory-maximum-limits-scalability(v=ws.10).aspx#BKMK_Objects<\/a>
<\/em><\/span><\/p>\n\n\n\n

[2.2]: The first row introduced in the “datatable” isn’t a real object nor is it a phantom and is named “$NOT_AN_OBJECT1$” and have its PDNT_col set to NULL.
The “PDNT_col” is indexed so becomes very easy to drive an object’s direct-descendants\/child objects (not all descendants) by simply query who that has the object’s DNT in their PDNT_col.
<\/em><\/span><\/p>\n\n\n\n

[2:3] The DSA has an in-memory cache for the most common RDNs (the ones mentioned above)
Active Directory allows us to use a custom attribute as RDN as well by specify the attributeID of that custom attribute as the rDNAttID<\/a> for the particular class. * A RDN attribute must have the syntax string **customization may not be supported by all LDAPv3 clients. *** rDNAttID should preferably be set before any objects of the given class is instanced in the directory (changes won’t apply to already instanced\/existing objects)
<\/em><\/span><\/p>\n\n\n\n

Let’s apply “Table 2” to a theoretical sample:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

[3.1]: “Name” represent the RDN attribute, it’s not stored\/named as described in the illustration above, it’s rather stored as “ATTm589825” I choose to represent it as “Name” for simplifying the understanding of the hierarchy in this case.
<\/em><\/span><\/p>\n\n\n\n

[3.2] Structural Phantom \u2013 (Different from a phantom used for reference integrity to real object’s hosted outside the given DIT) is used to represent the full distinguished name of the domain e.g “DC=ntdev,DC=corp,DC=chrisse,DC=com”
<\/em><\/span><\/p>\n\n\n\n

In the next article \u2013 we will continue the deep-dive into the content and the structure of the “datatable” \u2013 going thru things like ancestors, the difference between phantoms and real objects, tombstones and the garbage collector on the DBLayer and much more.<\/p>\n","protected":false},"excerpt":{"rendered":"

This is the second post in a series of articles that will describe what’s really inside NTDS.dit and how Active Directory works on the database layer, In an earlier post I explained the tables within NTDS.dit in detail as far as what they are used for, in which release of Active Directory (Windows Server) they … <\/p>\n

Continue reading “How the Active Directory \u2013 Data Store Really Works (Inside NTDS.dit) \u2013 Part 2”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[1],"tags":[6,14,16],"class_list":["post-781","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-active-directory","tag-extensible-storage-engine-ese","tag-ntds-dit"],"_links":{"self":[{"href":"https:\/\/blog.chrisse.se\/index.php?rest_route=\/wp\/v2\/posts\/781","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.chrisse.se\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.chrisse.se\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.chrisse.se\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.chrisse.se\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=781"}],"version-history":[{"count":2,"href":"https:\/\/blog.chrisse.se\/index.php?rest_route=\/wp\/v2\/posts\/781\/revisions"}],"predecessor-version":[{"id":1077,"href":"https:\/\/blog.chrisse.se\/index.php?rest_route=\/wp\/v2\/posts\/781\/revisions\/1077"}],"wp:attachment":[{"href":"https:\/\/blog.chrisse.se\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=781"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.chrisse.se\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=781"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.chrisse.se\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=781"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}