Clarion and the War on Terror

by Ozzie Paez

Published 2005-10-20    Printer-friendly version

Developing applications for a stable environment and developing applications to support research can be as different as developing with Clarion and non-RAD tools. Traditional application development allows the programmer to define most key requirements up front, validate and adjust them along the way, test and deploy within a generally known set of conditions. Developing in support of research, however, often involves at least as many unknowns as knowns, since the research itself is bound to affect initial assumptions and considerations.

This article discusses Clarion’s use in support of Homeland Security research conducted over the past two years, and Clarion’s demonstrated advantages supporting evolving research. This experience represents a specific case within the evolution of a programming paradigm, which I presented at a DevCon 2004 session titled Evolving Today’s Development Paradigms to Assure Software Lifecycle Quality and Security in Cyberspace. That presentation focused on how Clarion’s approach can be adapted to address many of the challenges facing programmers in developing quality products, within a highly dynamic environment. I will summarize this topic in the second part of this article; first, some background on how Clarion’s RAD capabilities have been useful in the War on Terror.

Background

In the spring of 2003 I was asked by a colleague to support a number of Homeland Security research projects being conducted through the Network Information and Space Security Center (NISSC) of the University of Colorado at Colorado Springs (UCCS). The research included evaluations of existing systems that could be adapted to various aspects of the War on Terror, as well as some initial modeling of the type of information required to conduct security threat assessments.

One of my deliverables during that time was a preliminary model that would illustrate a conceptual information management system to support these efforts. The research conducted during this phase included potential targets, mode of attacks, logistical/costs requirements and a variety of other factors, which were expected to play a part in the enemy’s decision making process.

The initial NISSC project evolved into additional studies focusing on integrating security risk assessments into Environmental, Health and Safety programs. The results of these efforts, which included information modeling, was a Security, Environmental, Health and Safety assessment model/process, which was piloted through additional grants. The results of these studies were later published in Environmental Quality Management (Spring 2004) and Disaster Prevention and Management (Volume 14/No 1/2005, UK).

Supporting research into what was then an evolving practice meant that data and information models were essentially in a constant state of flux. Their evolution was driven by a combination of changing use cases and improved understanding of the factors affecting the enemy’s thinking. The following sections describe the efforts involved in more detail, and the role which Clarion’s highly integrated RAD environment played in making it possible to quickly respond to the highly dynamic nature of the underlying data and user interface requirements. Clarion contributed greatly to the delivery of targeted solutions, from utilities to conceptual designs, that supported our work with minimal disruptions from undocumented features, i.e. bugs.

Terrorism and Databases

Terrorism did not begin with 9/11; instead, that awful day brought much greater awareness of the threats we faced and the real urgency of our response. From an information management perspective, a need immediately arose to capture as much information as possible about the terrorists and their tactics, methods and strategy.

One of the most challenging aspects of the war on terror has been to identify the threats posed by the enemy, without putting at risk the civil rights of individuals based on group factors, i.e. race, nationality, religion, etc. Researching threat profiles from radical Islamist organizations meant going back in history to identify the common characteristics of the various movements, which ultimately evolved into what is loosely referred to as AlQaida. Specifically, the research focused on these organizations’ membership, their behavioral characteristics, demonstrated capabilities, choices of weapons, means of attacks, strategy, tactics and other factors, which if properly analyzed could assist national authorities in identifying and neutralizing threats before they are carried out.

These efforts attempted to understand how the enemy made decisions, and how those decisions culminate in a resultant event, such as the attacks against the USS Cole, the Embassy bombings in Africa, two attacks against the World Trade Center and similar events across the globe, as we recently witnessed in London.

From an information management perspective, databases had to be created to contain the data uncovered during research, the application of models and the results of pilot programs. Data sources included published reports, Internet sources, and interviews with various government representatives and original research.

Diverse Sources and Dynamic Requirements

Designing a support system to facilitate research based on a diverse, and more importantly changing, set of parameters, all within a highly compressed timeframe, would stress any development team. The individuals with whom I continue to collaborate, and those whose work we have studied, are equally diverse in their areas of expertise and computer skills. These people include Environmental, Health and Safety practitioners, process analysts, experts in organizational leadership, historians, psych majors, statisticians and emergency response managers. Their professional backgrounds included academia, US military, corporate management, consulting and emergency response.

Such variety in the researchers and in the nature of the subjects being studied translates into constant change and adaptation of databases and related user interfaces.

Development Strategy

The nature of the dynamic environment and evolving requirements of these research projects precluded the development of full requirements definition in the traditional sense. There were initial efforts to create core data models and these proved useful, but they were just a starting point.

An analysis of the efforts to be supported and the dynamic nature of the data made it clear that Clarion’s Data Dictionary would have to serve as the nerve center of the effort. What made this possible, of course, is Clarion’s ability to generate code that implements dictionary requirements with minimal programmer involvement, provided that the dictionary is fully leveraged through detailed definitions of fields, default values, relationships and other parameters.

An interesting strategy for dealing with SQL backends under this development and operating environment was to minimize the work done at the database level, before the underlying architectural relationships were validated. Thus, keys and parent-child relationships were often defined and validated through the dictionary/application, before implementation was shifted to the database itself.

Where research tools were intended for use over a short duration, such as to demonstrate specific use-cases and underlying models, database relationships and constraints were kept within the Clarion application, saving valuable time. We didn’t have the time to define, develop and test backend components, i.e. triggers, stored procedures and views. Implementing, modifying and testing these components would have required us to take the database off line repeatedly, risk introducing coding errors, and significantly delay most deliverables.

By relying on the dictionary and the application, the desired behaviors (relationships, constraints, cascaded actions) could be easily implemented and the application updated and tested as requirements changed, without significantly affecting the underlying database and its availability.

Quality and Productivity

In almost all software engineering/programming situations, it is the programmer that introduces a majority of the errors. In the past, error rates were related to lines of authored code. Thus, for projects where the error rate was estimated as one per five hundred lines of authored code, a program with one hundred thousand lines of code should be expected to contain about two hundred errors.

Most errors are easily identified and corrected due to the manner in which they affect program behavior. Others, however, may not be identified unless a specific set of conditions, which might not be included in the test plan, exist. These are often the source of unexpected, nasty surprises, which almost all programmers have and will repeatedly experience during their careers.

Part of the strategy for managing dynamic change in support of the research described above was to limit the amount of embedded or hand-written code. In this environment, quality and productivity was provided by the general data architecture, the use of the dictionary and the template-generated code. The dictionary became the heart of the research support system precisely because it reflected the underlying data model(s), which in turn reflected the environment, organizations and people being studied. The dictionary actually became an important research tool in itself, by revealing relationships and conflicts between various aspects of the data and information being captured. As a result, it served as a tool that helped frame the information/data being gathered in a coherent and consistent manner.

In the end, the strategy described above resulted in very high productivity and quality, measured as the ability to quickly adapt the underlying data architectures and screen designs based on new information, requirements, distinctions and user needs. Software bugs were relatively few and did not significantly impact system availability and usability. Since the screens themselves were often all or part of the deliverables, use-case methodologies were defined and implemented through Clarion’s Designer, in order to contextually present the underlying data/structures in different ways for different audiences. Thus, not only were systems and tools quickly created, Clarion’s productivity allowed us to demonstrate different user interface designs in to support varying audiences. For example, one person might be studying high-end threats such as Weapons of Mass Destruction, while others may need to drill down within a particular type of WMD to the logistics, knowledge, skills and other factors associated with its construction, deployment and use.

Lessons Learned for Quality and Security

Clarion for Windows proved capable of supporting research within a highly dynamic and difficult to predict information management system design environment. Its ability to quickly translate data models into specific table designs and overlay relationships, constraints and behaviors based on dictionary directives proved ideal for this purpose. Since almost all development was against SQL based databases (MSSQL and Oracle), using the dictionary to prototype and validate the architecture, relationships and constraints saved countless hours.

In parallel, Designer’s ability to translate dictionary directives into user interface designs and behaviors, combined with its advance code generation capabilities proved itself in the implementation of client server and web-based systems and tools. At a higher level, templates were leveraged as much as possible to minimize hand coding. Where appropriate, new templates were obtained, designed or existing templates modified to implement recurring behaviors.

The lessons learned from the research projects described above support the concept of evolving traditional programming paradigms to better cope with today’s many technical and business challenges. These challenges include the ability to deliver better solutions, with higher quality, greater flexibility and improved cost-benefit levels.

As I write this article, it is being reported that Microsoft expects to outsource over ten thousand programming jobs to China over the next decade. While no one I know is seeking to keep the benefits of technology away from other societies, neither should we remain complacent as a growing number of companies work to commoditize and export a growing share of the most creative and critically important technology jobs in our economy. Exporting jobs is actually a two edge sword because, while on the one hand a company can lower per-employee unit costs, on the other communication between the worker and client (internal or external) almost always suffers, particularly over long distances, many time-zones and a wide cultural divide. Poor communication remains a critical root cause behind poor program design through poorly defined, understood and implement user needs; thus, real incentives exist for development to remain as close to the customer as possible.

The Clarion paradigm can successfully integrate evolving development paradigms with effective communications and complimentary programming team skill sets to offset the blind drive towards cheaper labor being promoted within some business circles. In the end, most clients do not want to know how much employees get paid per hour. What they want are solutions that address their needs, within a competitive cost-structure and with the quality necessary to ensure reliability and security.

For many areas in Government, security and quality are even more important, particularly, where applications are accessed through Intranet, and increasingly Internet, portals. Microsoft has been miraculous in its ability to deliver products to market, but that does not mean that it has a corner on ideas and concepts. And, when it comes to delivering quality applications, productively and competitively, the Clarion paradigm offers advantages that have proven effective in supporting the national efforts against terrorism. Clarion can prove even more effective in growing future opportunities for the next generation of American programmers and solution providers.

---

Ozzie Paez is a systems engineer with over 25 years experience as a programmer and database architect using a variety of languages including Fortran, Modula 2, Pascal, Basic, Assembly and Clarion. He has been using Clarion since 1987 and became a Certified Clarion Developer in 1999. Mr. Paez's work history includes the US Military, Civil Nuclear industry, Department of Energy's Weapons Complex, Telecommunications, Technical Support and Security Center Management. He currently works as a systems engineer, solutions provider, software quality assurance lead and security researcher for a variety of clients, including the US Air Force. He has published a variety of articles in the US and Europe on issues ranging from business management, corporate leadership, regulatory compliance and quality, to his current focus developing models for conducting threat assessments in the war on terrorism.

Printer-friendly version

Copyright © 1999-2008 by CoveComm Inc. All Rights Reserved. Reproduction in any form without the express written consent of CoveComm Inc., except as described in the subscription agreement, is prohibited.

Clarion Magazine ISSN 1718-9942