Handling Circular References in C7

Posted March 20 2009

Printer-friendly version

NOTE: As of C7 gold, build 5615, the IDE is capable of compiling solutions containing multiple DLLs with circular references. Use the Generate and Build button in the Application Pad, and repeatedly build the solution until all apps compile.

If you're one of those developers who indulge in circular references you may be in for a surprise when porting apps to Clarion 7, at least if you import your App files into a multi-project solution.

A circular reference is where you have a multi-DLL application and some code in DLLA calls a procedure in DLLB, and vice versa. Since statically linked DLLs (which is what most of us use) require the DLL's LIB file to be linked in, you have to compile the DLL (which creates the LIB) before you can link the application. When you have circular references you have a which-came-first-the-chicken-or-the-egg problem; the solution is to do multiple compiles until you build up all the LIBs you need for the entire application and you no longer get compile errors.

(Can you have a circular references in your apps and not know it? Yes you can, if you've never cleaned out all your LIB files and tried to compile from scratch. The only warning you'll get from C6 and earlier versions of Clarion is via missing LIBs.)

Circular calls and C7

Circular references are potentially more of a problem in C7 than in C6 because of C7's ability to create solutions containing more than one project.

When you import a multi-DLL app into C7 and you tell the IDE you want to import any related apps, the IDE creates one solution containing all the related APPs, and it creates the relationships between APPs as project references. Essentially this tells the IDE that App A references some code in App B, and that makes it possible for the build system (which is based on MSBuild) to assemble all the needed bits (usually LIB files) so the compile can happen.

In C7, if you have circular project dependencies you'll be prevented from compiling the solution. That's because MSBuild doesn't tolerate circular references, and to fix this you will need to change all circular project references to library references. That's a one-time deal, and then you're good to go. Mostly.

Creating a circular call

Figure 1 shows the Solution Explorer displaying a C7 version of the DLLTutor application. I've added a button to the ViewCustomers procedure in Updates.app, and from that button I'm calling CustReport, which is in Reports.app. When I generated the code, the AppGen automatically inserted the project reference to Reports.

Figure 1. The DLLTutor example with a circular call

Updates now needs Reports.lib to compile, and Reports needs Updates.lib to compile. But the IDE won't let you get that far. If you attempt to build the solution you'll see this error:

Dependency cycle detected, cannot build!

That message isn't as helpful as it might be, since it doesn't tell you which apps are involved. If you only have a few apps in the solution you can eyeball the problem, but if you have dozens of apps a clue of any kind would help. To get that clue just run MSBuild from the command line. On my machine the command to build the DLLTutor solution is as follows (line breaks added):

C:\Windows\Microsoft.NET\Framework\v2.0.50727\msbuild 
   dlltutor.sln /property:ClarionBinPath= 
  "C:\Program Files\SoftVelocity\Clarion 7\bin"

(You can read more about command line compiling in my article Building C7 Apps With MSBuild.)

MSBuild responds with the error:

MSBUILD : error MSB4006: There is a circular dependency  
  in the target dependency graph involving target "reports".

Although the problem is due to the added call in Updates which resulted in a new project reference to Reports, you can fix the problem in either Updates or Reports by changing the project reference to a library reference. Library references don't enter into MSBuild's determination of circular dependencies.

Since MSBuild is reporting the problem in Reports, you might as well fix it there. As shown in Figure 2, right-click on the Updates project reference under Report and choose Remove.

Figure 2. Deleting a project reference

Next, right-click on the Libraries, Objects and Resources folder under Reports and choose the Add option. You'll be presented with a file dialog; navigate to the obj\debug or obj\release directory (depending on whether you're in debug or release mode) and choose the Updates.lib file. You should end up with a library reference as shown in Figure 3.

Figure 3. Adding a library reference

If Updates.lib doesn't exist, you'll first have to go back to the Solution Explorer, right-click on that LIB's app and choose Build. Then you'll be able to pick the lib from the Add Library Reference file dialog.

Once you have all the circular references fixed up you'll be able to build the application. And you'll have all the LIB files because you just went through the process of manually building them from the bottom of the hierarchy, which is what you were actually doing in the previous paragraph.

When I was testing all of this code out I ran into a puzzling situation. During one of my test cycles I introduced a circular reference as described above. I deleted the project reference, but I forgot to create the library reference. And each time I generated the code, the AppGen recreated the project reference. The moral of the story: AppGen creates references as needed, but it won't replace a library reference with a project reference.

Rebuilding from scratch

What if, after you have your solution built, you want to rebuilt the solution from the ground up? That is, you delete all of the LIB files and you try to compile. If you have circular dependencies you'll once again get a compile error telling you that such and such a LIB is missing. Building repeatedly accomplishes nothing; MSBuild stops at the same point every time.

You can, once again, manually build individual libs as needed in order to proceed. Or you can automate the process.

Multi-pass builds

In a comparable situation in C6 you would use a batch compiler and simply run the process twice (or as many times as necessary to create all the needed LIBs). Unfortunately, in C7 compiling a solution is a single task, and by default if MSBuild encounters an error it aborts the task.

The way to get MSBuild to continue compiling despite errors is to create a specific kind of task also called MSBuild, and call that task directly.

Let me explain that. MSBuild (the EXE) is the tool you use to build a project or solution. And if you execute MSBuild (the EXE) from the command line, you normally pass it either the .sln file or the .cwproj file for the solution or project (respectively) you want to build.

You can, however, create a specialized project file containing build tasks. MSBuild ships with a number of standard tasks, one of which is, like the build tool itself, called MSBuild.

The way to get MSBuild (the EXE) to compile a series of projects without stopping on errors is to use MSBuild (the task) with the ContinueOnError attribute set to true. Here's a project file that does just that for the DLLTutor projects:

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Circular"  
  xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <Target Name="Circular">
      <MSBuild Projects="allfiles.cwproj;reports.cwproj; 
        updates.cwproj;dlltutor.cwproj" Targets="Build"  
        ContinueOnError="true"/> 
    </Target>
</Project>

Save this to a file called dlltutor.msbuild, and execute it like this (changing paths as necessary):

C:\Windows\Microsoft.NET\Framework\v2.0.50727\msbuild  
  dlltutor.msbuild /property:ClarionBinPath= 
  "C:\Program Files\SoftVelocity\Clarion 7\bin"

But that project only executes the MSBuild task once. To do two passes, simply duplicate the MSBuild task:

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Circular"   
  xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <Target Name="Circular">
      <MSBuild Projects="allfiles.cwproj;reports.cwproj; 
        updates.cwproj;dlltutor.cwproj" Targets="Build"  
        ContinueOnError="true"/>
      <MSBuild Projects="allfiles.cwproj;reports.cwproj; 
        updates.cwproj;dlltutor.cwproj" Targets="Build"  
        ContinueOnError="true"/> 
</Target>
</Project>

Note that the two calls to the MSBuild task are identical; they result in two separate compiles for each of the project files listed, and by the end of the second pass everything's successfully built.

There are other ways to declare the MSBuild task. Here's one that uses a wild card approach to build all C7 projects in the current directory in two passes:

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Circular"  
  xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <ItemGroup>
        <ProjectsToBuild Include="*.cwproj"/>
    </ItemGroup>
    <Target Name="Circular">
        <MSBuild Projects="@(ProjectsToBuild)"  
          Targets="Build" ContinueOnError="true"/> 
        <MSBuild Projects="@(ProjectsToBuild)"  
          Targets="Build" ContinueOnError="true"/> 
    </Target>
</Project>

There are lots of other ways to get a list of projects to build; MSBuild is both flexible and powerful.

Project references vs library references

You might be tempted to just drop project references in favor of library references, but there are consequences. When you exchange a project reference for a library reference MSBuild loses that dependency information, which is why you can build a solution with circular references. But if you remove a non-cyclical project reference you are potentially depriving MSBuild of useful information. You could, for instance, change all project references to library references, and then MSBuild wouldn't have any idea of an appropriate build order. The way around this is to use the Dependency Editor, as described by Steve Parker in C7 And The Demise Of The Batch Compiler.

But really, the best solution is to get rid of your circular calls in the first place. Yes, I know, you've been using them for years without any problems. But heed the words of George Lehman in Eliminating Circular DLL Calls:

For reasons I don't fully understand but which are suggested to be linked to the new threading model, this picture evidently changed radically for the worse with Clarion 6.x. It seems now that mutually dependent DLLs can wreak major havoc with memory de-allocation, particularly at thread shutdown time, resulting in various "memory could not be read/written" errors during program execution. Bob Zaunere, in a news group posting on 4/29/05, stated "If DLLs are mutually dependent, the order of calls to their thread detaching code is undefined at the OS level. ...instances of data in the first DLL can be killed before the destructors using them are called in the second DLL." Sounds pretty bad to me.

Summary

If you find yourself with a circular dependencies in C7, here are the main points to keep in mind:

• If you get a circular reference warning, and the cause isn't obvious, run MSBuild on the solution, from the command line, to get a clue as to the offending apps.
• Ideally, remove all circular calls from your apps. George Lehman's article is a good place to start.
• If you can't remove a circular call, replace the project reference with a library reference.
• To get around compiler warnings on circular library references either build the required library's project via the Solution Explorer, or execute a batch build using an MSBuild task with the ContinueOnError attribute set to true.

Clarion 7's multi-APP solutions make it easier to work with large multi-DLL applications, but more difficult to accommodate circular calls. Your best option is to remove the circular calls; if that isn't possible, I hope you'll find these mitigation techniques helpful.

For further reading, here are a couple of interesting MSBuild links via Larry Sand:

• Best Practices For Creating Reliable Builds, Part 1
• Best Practices For Creating Reliable Builds, Part 2
• Book: Inside the Microsoft Build Engine

Download the source

Article comments

Post a comment

You must be logged on to post comments.