| IAC Project Organization Guidelines |
 |
Introduction | |
This document provides guidelines for the construction of AVS/Express projects. The motivation for this document is to encourage consistent and orderly organization of AVS/Express projects and modules, for submission to the International AVS Centre's public domain module repository.
If these guidelines are followed in the construction of AVS/Express project, you should find it both easy to share your projects with other AVS/Express users, and to include other user's projects within your own work.
It is assumed throughout this document that the reader is familiar with AVS/Express; specifically its project structure, V code, and the process compilation procedures.
To simplify the integration of the IAC projects into the user's own projects, the IAC Project Files must be placed into the user's project directory. Once added, the individual IAC projects may be downloaded and added to the IAC project structure. This document describes the structure of the IAC project files, how new projects are added, and lists which parts of the project should be submitted.
To demonstrate the project organization, an example project is presented throughout the document. The project is a simple implementation of a threshold function, and to avoid confusion with the existing AVS/Express threshold module/macro, it is referred to as IAC Threshold throughout this document. The IAC Threshold module performs a simple threshold algorithm upon the Node_Data of a Field, allowing the user to specify an upper and lower bound. Further details about each object within the IAC Threshold project are provided in this document. The IAC Project Files must be downloaded and setup first.
|
Project Architecture | |
Any project created within the AVS/Express environment will contain a subset of the principle objects outlined in this document. By following this arrangement, a consistent model can be easily enforced upon each project. It is strongly recommended that you base your project organization on the model presented here, and you must do so, if you wish to submit projects to the IAC.
Not all projects will fall into the same category as the IAC Threshold module presented here. Some projects may only consist of new group definitions, or contain modules which solely use V code to perform a simple task. Additionally, users may want to submit new macros which use existing components from AVS/Express. In these instances the submitted projects need only contain the principle objects which are present in the new project.
DV Module/Macro Architecture
The project model closely matches that used by the Visualization and Template.DV libraries in AVS/Express, which provides some of the flexibility and consistency needed for successful organization.
The DV library model is composed of a number of objects, and closer examination of the macros such as those found in the Filters and Mappers libraries, reveals that they generally consist of the following types of objects. Note that these component objects may also be found independently within the template libraries.
- Parameter Block
This provides a convenient way to connect a user-interface to the parameters of a module. Some of these can be found in the Templates.DV_MACROS library, e.g., DV_Param_clamp.
- Module
The low-level module performs an operation on a minimal set of data. The module will also include methods, which are invoked when the data changes, and may be in the form of V code, C functions, Fortran functions or C++ classes. Some modules may be found in the Templates.DV library, e.g., DVclamp.
- Visualization Macro
These macros contain all the necessary objects to process a higher-level datatype, e.g., Fields, but do not contain any user interface. The low-level modules described above, may only process part of a Field object for example. Hence they are encapsulated by these macros, so that all modules operate on a unified datatype (Field in this example). Some visualization macros may be found in the Templates.DV_MACROS library, e.g., Clamp.
- Encapsulated macro
This contains an instance of the parameter block, the visualization macro - which in turn contains the low-level module(s) - and a suitable user interface macro. These are the macros found in the Filters and Mappers libraries for example. The user interface will attach itself to the application (e.g., the module menus of the SingleWindowApp and MultiWindowApp viewers.)
The model presented in this document extends that definition, so that it may be applied to more generic modules/macros. For example, the user interface macro found inside the encapsulated macros, is not normally found within the templates, but still exists as a separate object.
|
Principle Objects | |
The objects described above may be further generalized, to clarify the requirements of the project organization. The following set of principle objects will be encountered within a project. As each object is identified, the IAC Threshold project is given as an example.
Primitives
The lowest level of all objects are called primitives, and are objects of type module or group (depending on whether or not they include procedural code).
A module takes only a minimum amount of information necessary to perform its function. Many of the primitives found in AVS/Express operate on only a part of a Field structure. For example, the DVthreshold primitive operates only on the Field's Node_Data. As such it does not affect the Field's Mesh, and does not require a mesh to be present on its input.
|
Shows the primitive module ThresholdNodeData as it would appear in the Network Editor. |
A group provides a convenient way to collect a number of objects (data types) together into one place. The most common use of this is the specification of the parameter block as used by the module and the associated macros.
|
Shows the primitive parameter block group ThresholdParams as it would appear in the Network Editor. |
Functional Macros
The next level in the hierarchy encapsulates the primitive modules and groups (parameter block) into a macro. These are completely functional modules and do not include a user interface.
They serve two purposes: allowing the user to build a custom user interface, if the user interface provided in the User Macro (see below) is not suitable, and secondly, the lower level primitives can be packaged in such a way that the macros may be placed in a library such as the Visualization Macro library, where all the macros input and output Fields, making them all consistent and easy to combine.
|
Shows the high-level Functional Macro Threshold as it would appear in the Network Editor. |
The macro contains the primitives, but also other modules such as combine_mesh_data, which are used to reconstruct the data into the unified type, e.g., Field.
UI Macros
Alongside the Functional Macros, the UI macros simply contain objects from the user interface kits provided in AVS/Express. This macro is provided for macros/modules that require a user interface, and allows a suitable interface to be attached to the functional macros, via the common parameter block.
The user may now use the existing UI macro, edit it if need be, or not use it at all, opting to create an entirely new interface.
|
Shows the high-level UI Macro ThresholdUI as it would appear in the Network Editor. |
The DV libraries do not have an independently stored UI Macro, and it is not entirely necessary to do so, but it does add more flexibility to the model, and makes it easier to manage. Using this approach would allow multiple UI Macros to be provided if desired, and editing of existing UI Macros is made simpler if it is defined separately.
User Macros
The next level in the hierarchy combines all of the objects described above, producing an object which can be placed into the libraries such as the Filters and Mappers in the Main library. User Macros encapsulate the Functional Macros and the UI Macros into a single immediately usable macro; when they are instanced the user interface is automatically displayed. The User Macro also contains an instance of the parameter block, which is referenced by the other macros.
This is the level of objects required by most users of AVS/Express. With User Macros, networks can be quickly developed, as they share common connectivity (e.g., Fields), inherited from the unified type of the Functional Macro, and also include an appropriate user interface.
|
Shows the high-level User Macro threshold as it would appear in the Network Editor. |
Example Applications
Finally, for any projects that are developed, it is a good idea to include an example of how the objects defined above, could and should be used. An example Application should include at least one instance of the User Macro, which in turn utilizes the functional macros, the UI macro, and the parameter block. The examples are likely to be instances of AVS/Express Applications, such as the SingleWindowApp or MultiWindowApp viewers.
The example application will (probably) include other high-level macros to support it, to create a suitable demonstration. Ideally, any additional objects should be those already supported by AVS/Express, rather than using other externally developed macros, unless this is unavoidable. If a number of macros/modules are to be submitted together as a package, then the project should contain all of the required objects.
|
Shows the high-level Example Application ThresholdEg as it would appear in the Network Editor. |
Using the Objects
ThresholdNodeData operates only on the Node_Data part of an AVS Field. Its parameters are found in the instance of &ThresholdParams, but this is only a reference to the actual parameters that are outside of the module. Similarly, the ThresholdUI macro contains a reference to a parameter block of the same type. As can be seen in the User Macro threshold, both the Functional Macro Threshold and the UI Macro ThresholdUI contain group references to the same parameter block ThresholdParams that is present in the User Macro.
The user interface objects may now modify members of the parameter block, so when the user manipulates a widget in the user macro's interface, a member of the parameter block is changed, causing the referencing primitive module to execute its method(s).
This hierarchical arrangement of objects has a number of benefits to both the developer and the user:
Minimum Specification of Inputs
Primitive modules specify their inputs as minimally as possible. This provides flexible (loose) type matching, allowing a single module to operate on a variety of unified datatypes, e.g., the numerous variations of Field.
Consistent Interchangeable Objects
Functional and User Macros are all "complete" visualization objects which input and output a common type. This makes them all interchangeable in visualization networks, increasing the ease of use of the macros.
Configurable User Interface
The user interface for a module is kept separate from the module's implementation, in the UI Macro, allowing for easy replacement, modification or deletion of the interface. Separation also enables multiple-process applications to be built that share a single user interface. Due to limitations in the X Toolkit, all user interface components in the same "user interface hierarchy" (essentially displayed in the same user interface shell) must reside in the same AVS/Express process. Separation of the user interface from the module means that the user interface for several modules can be collected together in a single process, even if all the modules themselves execute in multiple processes.
User Interface uses UImod_panel
A user interface is not necessary for every module/macro. If you do choose to provide a user interface with your module, it is recommended that you follow the style of the ThresholdUI UI Macro described above.
The key component of the UI Macro is the use of the UImod_panel object from the User Interface library. This is a special version of the UIpanel object that interacts automatically with the top-level Applications. When instanced, a UImod_panel will attempt to "install" a UIoption toggle representing the UImod_panel in the Modules pull-down menu, of the SingleWindowApp and MultiWindowApp applications. When this toggle is selected, the UImod_panel will become visible in the viewer's window.
Examination of the contents of ThresholdUI reveals an object named UIpanel. Performing an "info" operation on this object in the Network Editor will show that its type is UImod_panel.
Basing the user interface of the UI Macros on a UImod_panel, makes the registration behaviour automatic. This will make the UI Macros behave in a fashion that is consistent with the built-in Visualization Macros, and those submitted by other users.
Note that the title of the UImod_panel defaults to "name_of(<-)", which would make it appear as "UImod_panel". In the UI Macro, the title may be set explicitly, as the IAC Threshold project does, but in the User Macro this should be changed, so that the title changes to the name of the instanced user macro (e.g., "threshold#1"). To do this, the UImod_panel of the UI Macro instance in the User Macro is modified, setting title to "=> name_of(<-.<-.<-)" which will use the name of the UImod_panel's grandparent; the User Macro instance.
|
Libraries | |
Libraries provide a central object on which many properties can be set, which are inherited by the member objects. This simplifies management, code generation, compilation and the exchange of projects. It is recommended that the AVS/Express flibrary object is used to organize the modules and macros in any project.
IAC Project Files
To make the integration of submitted modules into a user's project easier, a number of IAC Project Files must be downloaded and setup, before any projects can be added. This only has to be done once, for each of the user's projects, that need to use IAC projects.
The project files create the following structures: 1) a Templates.IAC_PROJ library, into which the individual projects are added, 2) a Templates.IAC library, which contains seven project category libraries, and 3) a top-level IAC library page, which links to the category libraries.
Templates
As already described above, it is desirable to separate the module and its user interface into different processes. This can be easily accomplished by placing the primitives into one library, and the higher level macros (including the UI Macro) into another library. This is also a useful way to organize the structure and storage of the various objects in each project.
In the IAC Threshold example, the low level primitives (ThresholdNodeData and ThresholdParams) are placed into a single library called ThresholdMods. Similarly the high level macros (ThresholdUI, Threshold, threshold and ThresholdEg) are placed into the library called ThresholdMacs.
Each submitted project should place both the low-level primitives library and the high-level macros library into another library named after the project. This results in a single library which contains all of the submitted project's components. This library is subsequently placed into the Templates library.
 |
Shows the hierarchical organization imposed on the IAC Threshold sample project, as it would appear in the Templates.IAC_PROJ library in the Network Editor. |
The IAC Threshold project has a top-level library called Threshold, stored as Templates.IAC_PROJ.Threshold. This contains the low-level module and the high-level macro libraries, stored as Templates.IAC_PROJ.Threshold.ThresholdMods and Templates.IAC_PROJ.Threshold.ThresholdMacs.
The macros in the high-level library will contain instances of the primitives from the low level library. Each library can specify a different process, for example the high-level library's process will by default be express. Instances of the primitives in the high-level library macro in which they were instanced, will however run in the process specified in the low-level library, and not the process of the high-level library.
Network Editor Libraries
The Templates.IAC library contains seven category libraries, into which the User Macros and Example Applications from the individual projects can be referenced. This in conjunction with the top-level IAC library page, provides a useful way to present these objects to general users.
The IAC Threshold project is categorized as a Filter, and is added to the Templates.IAC.FILTERS library, and is then visible in the IAC.Filters library. Similarly the ThresholdEg Example Application is added to the Templates.IAC.EXAMPLES library, and is visible in the IAC.Examples library. All the principle objects are still accessible via the Templates.IAC_PROJ hierarchy.
V Code Organization
The combination of the project library hierarchy and the IAC Project structure, results in a hierarchy of V code use to define all the objects. So that a number of projects can easily be added to an existing user project, the V code that defines the primitives and the macros is stored in separate V files, in an appropriate directory.
Using the Project->Save Project menu option within AVS/Express would normally store all of the V code in the templ.v file. This makes it difficult to maintain and organize for submission. The low-level modules and the high-level macros can be defined in their own V files; <nam>_mods.v and <nam>_macs.v.
To overcome the problem of AVS/Express storing all the V code for each module in the templ.v file, one or more of the objects can be selected and saved to a specified V file. For example, the definition of the ThresholdParams group and the ThresholdNodeData module can be selected (hold down SHIFT to select more than one) and then using the Object->Save Object menu option, their definitions can be written to a file; in this instance, thr_mods.v. The high-level macros are similarly stored in thr_macs.v.
This can be shown more clearly in the following hierarchy of V files for the example IAC Threshold project:
The files indicated in bold refer to those that are created as part of the initial IAC Project Files structure. It is important to note that the files templ.v, proc.v, lib_xp.v and lib_vxp.v are not changed by the addition of newly downloaded projects. In particular, the templ.v file need only contain the two lines to reference the iac_proj.v file which adds individual projects, and the iac_lib.v file to include the User Macro and Example Application references.
Note that the order of the files in the libraries is important: for the individual projects, the low-level primitives must be defined before the high-level macros which use them, e.g., thr_mods.v is before thr_macs.v.
The addition of a newly downloaded project requires just three edits: 1) a single line added to iac_proj.v to add the new project to Templates.IAC_PROJ under a unique library name, and 2) a single line addition to one of the category library files (common.v, data_io.v, filters.v, mappers.v, glyphs.v and misc.v), to reference the User Macro, so that it is visible in the top-level IAC library page, and 3) a single line addition to the examples.v file, to reference the Example Application, so that it too is visible in the top-level library.
The IAC Threshold example requires the addition of the following line to the iac_proj.v file:
"../iac_proj/thresh/thresh.v" Threshold;
To add the User Macro threshold to the Filters library requires the following addition to the filters.v file:
IAC_PROJ.Threshold.ThresholdMacs.threshold threshold;
Finally to add the Example Application ThresholdEg to the Examples library requires the following addition to the examples.v file:
IAC_PROJ.Threshold.ThresholdMacs.ThresholdEg ThresholdEg;
For submission this information should be given in a text file named INSTALL. The format of the file is such, that a future installation script will be able to read it. Users downloading the project will use the INSTALL file to determine what lines should be added to the various files.
Directory and File Organization
The IAC Threshold example presented here uses the following directory and file structure for the storage of the files:
<user-project-dir>/
v/
iac_lib/
iac_proj/
thresh/
The directory and files indicated in bold are those that are added for the IAC Threshold project. All other files are part of the initial IAC Project Files structure.
The thresh/ directory contains all the V code definitions and source code for the IAC Threshold project. Each submitted project should have all its code placed in such a directory, with a unique name (see later), which is subsequently placed into the iac_proj directory.
Apart from the changes to the iac_proj.v file and the iac_lib/ files, submitted projects should not require code to be placed anywhere else. Initially these edits should be made manually, although the IAC intends to eventually release some scripts which will automate the process.
It is recommended that the name of the directories within iac_proj/ should be the same name or at least similar to, the name of the project directory, for submission of the project to the IAC. The attribute build_dir is set to point to the iac_proj/projname/ directory for each submitted project.
It is likely that the names of some of the projects submitted will clash with other submitted projects. In such instances the IAC will attempt to resolve the clash.
Use Version 3.0 V Library Syntax
In version 3.0 and later of AVS/Express, the following flibrary syntax should be used to define object libraries:
"../iac_proj/<library_name>[.v]" LIB_NAME;
This syntax allows several library operations in the Network Editor that were not possible with the older library LIB_NAME<libfile="library_name[.v]">; syntax. It also makes it easier to merge libraries from multiple projects, requiring only a one-line change to iac_proj.v to include a newly acquired library.
Properties
In the course of building AVS/Express modules and macros for projects and applications, several object properties are required to control the configuration and behaviour of the project. There are specific recommendations regarding the use of these properties.
Use build_dir
It is strongly recommended that you use the build_dir property on the flibrary objects. Related source files may now be organized and placed into common directories, separating them from unrelated files. A short name should be chosen that is likely to be unique, remembering that AVS/Express runs on both UNIX workstations and PCs. The Windows 95 and NT platforms may still contain many PC tools for archival and backup which can still not handle filenames longer than the MS-DOS 8.3 format.
The value of the build_dir property should be related to the name of the library in a way that is obvious, so that a user examining the directory and V code would be able to associate the two easily.
Use of directory names that exist in the installed version of AVS/Express should be avoided. This includes the most common choice of src for the value of build_dir. Use of directory names that are already in use by AVS/Express can lead to confusing errors during compilation and linking.
The IAC Threshold project uses the iac_proj/thresh directory as the value of the build_dir property, and defines this solely in the low-level modules V definition file thr_mods.v. Any AVS/Express generated files and the compilation *.o files will reside here, once the project has been compiled.
Avoid Absolute Pathnames
Many properties in AVS/Express have values that are filenames or directory names. Absolute pathnames should never be used as values for these properties. The AVS/Express project mechanism determines pathnames automatically, with information from the avsenv file, using the XP_PATH environment variable, and from the XX_src_file and other properties. The presence of absolute pathnames in object properties will only lead to unnecessary difficulty in exchanging projects.
Avoid process Property Where Possible
In general, the use of the process property should be avoided. It is useful for separating modules into multiple processes. If the process property must be used, then it is recommended that it be set in the definitions for the low-level and/or high-level libraries, e.g., in thr_mods.v and thr_macs.v. Note that the IAC Threshold project does use the process property on the low-level primitives, to demonstrate its use, setting it to "user" and specifies it as such in the thr_mods.v file.
Avoid global attribute on Libraries
The global attribute is used on several of the standard AVS/Express libraries. The use of this attribute on built-in libraries eases the use of common objects like primitives types (float, int, etc.) The global attribute essentially collapses the hierarchical object name-space, which may lead to name clashes, hence it is recommended that the global attribute should be avoided when building the project libraries.
Note that the older type library has the global attribute set by default, but is now considered obsolete, and flibrary should be used instead, at least at the level directly below Templates.
Choosing to use the global attribute on libraries requires careful naming, and therefore it would more useful in this instance never to set any of the objects to be global.
Use out_hdr_file and out_src_file Properties
Various properties of objects in AVS/Express cause the generation of type definitions, class definitions, compiler directives, procedural code and other program fragments. The default location for all of these fragments is in the various generated files associated with the particular process which is being compiled, e.g., express.c, express.cxx, express.h and express.mk. In addition, when source code is generated via use of the src_file property, an include directive is placed within that source file to include the file that contains the function prototypes for functions present in the source code, e.g.,
#include "express.h".
If the process property on the library containing the V definition of the module that specified the src_file property is subsequently changed, the file named by src_file will NOT be regenerated. This will leave the #include "express.h" in the file named by src_file, but the necessary prototypes for the module will no longer reside in express.h.
The best way to avoid this problem is to use the out_src_file and out_hdr_file properties on the library. Examination of the IAC Threshold project reveals that these two properties are set on the low-level library, in the thr_mods.v file, causing all program fragments associated with these libraries to be generated into the gen.h and gen.cxx files, and the generated source files themselves will contain include directives of the form:
#include "gen.h".
Since the properties of the library will travel with it, when it is transported, this include directive line will remain valid no matter what happens to the process property. The generated files themselves should not be included with the project when it is submitted; AVS/Express will regenerate them when the new project is compiled.
Use cxx_name Property
When the module uses a C++ method, the resulting class names generated by AVS/Express will be based on the entire library hierarchy that leads to the module itself, e.g., the IAC Threshold module would (if it was a C++ module) produce class names of the form IAC_PROJ_Threshold_ThresholdMods_ThresholdNodeData, which is far too cumbersome.
The file iac_proj.v sets the cxx_name property to "", which removes the IAC_PROJ portion from the class name. The file thr_mods.v would again do likewise, so that the ThresholdMods portion would also be removed. The result is a class name based on the project's name and the module's name.
Identify Generated Files
Several properties in AVS/Express cause files containing code and other information to be generated. These files need not, and should not, be exchanged along with a project because they can be regenerated by the recipient of the project. As seen in an earlier section, two of the properties that cause code or other file generation are out_hdr_file and out_src_file. Use a consistent scheme to identify generated files when you set the values of these properties, and it will then be easier to remove the generated files when packaging a project for submission to the public archive or transmission to a colleague. Use filenames of the form:
<filename>_gen.<extension>
where "_gen" indicates a generated file.
The IAC Threshold project uses this convention for the generated files created by the low-level modules library. The file thr_mods.v specifies the generated files to have the names gen.h and gen.cxx. Note that all the filenames conform to the MS-DOS 8.3 format. Other generated files that need to be included should add the prefix to the _gen name.
Use notify_inst on "update" Method
The module ThresholdNodeData uses notify_inst on its omethod "update". This should be used on all typical modules' "update" methods. This is required when a network is loaded, which has valid data already within it, or it automatically loads a file: if notify_inst was missing, then the module would not be executed.
|
Project Documentation | |
Documentation should always be provided with a submitted project, and this should include general information about the purpose and use of the project and its component modules and macros, details about the data types for which it is intended (for example, it may only work with uniform fields), and any special notes about the process and global properties if they have been set to particular values.
Additionally any information about the author is useful, along with contact information, if the author wishes to receive comments and suggestions about future updates.
For submission of a project to the IAC it is recommended that the following HTML templates are used for the low-level Module and the high-level User Macro. The templates are based on the module and macro descriptions found in the AVS/Express documentation.
The IAC Threshold documentation for the ThresholdNodeData modules and the threshold User Macro, use the templates provided.
|
Submission | |
With the IAC Project files in place, the process of submitting a new project is easier. Only the iac_proj/projname/ directory and its contents should be submitted, either as 1) a tar.gz archive, 2) a tar.Z archive, or 3) a zip archive. The IAC will make all three archive formats available for download.
The source code must have used the conventions outlined in this document, so that it will merge without difficulty with the existing IAC Project Files. The submitted project must include the README file, outlining the project and its associated files, and the INSTALL file, indicating which components fit into which category; the IAC will create a suitable INSTALL file if one is not provided.