Comments on: Input Validation and Data Dictionaries http://www.cigital.com/justice-league-blog/2010/07/21/input-validation-and-data-dictionaries/ Sun, 13 May 2012 16:44:00 +0000 hourly 1 http://wordpress.org/?v=3.3.2 By: Tom Van Vleck http://www.cigital.com/justice-league-blog/2010/07/21/input-validation-and-data-dictionaries/#comment-191 Tom Van Vleck Thu, 23 Sep 2010 15:50:42 +0000 http://www.cigital.com/justiceleague/?p=408#comment-191 I worked at Tandem in the 80s. They had their own SQL database implementation, and a company-wide data dictionary. Using this dictionary exposed several problems with data dictionaries. 1. Administration. If there is bureaucracy involved in defining data elements, developers will bypass it. 2. Merging. When independently developed data dictionaries must be merged, either you end up rewriting applications or you end up with multiple similar fields. 3. Semantics. This is the killer. Simple example: we had an "airport" field in the corp data dictionary definition for a branch office. Some applications used it for the air freight calculation; others for expense report mileage calculation.. but some airports didn't do air freight, and the apps conflicted in their use. Each app designer said, "airport, aha, that's what I need" and didn't realize there was a semantic problem. Getting precise semantic definitions of each data element required people to model the whole real world, not just the part their app touched. You might be able to model what a mail address looked like (wonderful regex!) but the DD also has to have semantics for how mail addresses are related to other entities and how they are used. My SQL address book has about 5 email address fields and i find myself punning all the time to account for idiosyncratic uses. I worked at Tandem in the 80s. They had their own SQL database implementation, and a company-wide data dictionary. Using this dictionary exposed several problems with data dictionaries.
1. Administration. If there is bureaucracy involved in defining data elements, developers will bypass it.
2. Merging. When independently developed data dictionaries must be merged, either you end up rewriting applications or you end up with multiple similar fields.
3. Semantics. This is the killer. Simple example: we had an “airport” field in the corp data dictionary definition for a branch office. Some applications used it for the air freight calculation; others for expense report mileage calculation.. but some airports didn’t do air freight, and the apps conflicted in their use. Each app designer said, “airport, aha, that’s what I need” and didn’t realize there was a semantic problem. Getting precise semantic definitions of each data element required people to model the whole real world, not just the part their app touched.

You might be able to model what a mail address looked like (wonderful regex!) but the DD also has to have semantics for how mail addresses are related to other entities and how they are used. My SQL address book has about 5 email address fields and i find myself punning all the time to account for idiosyncratic uses.

]]>