Packed Project Libraries – Part 2
My inbox has been filling up with emails from those of you who have been eagerly awaiting this follow-up to the previous post. Okay, so the truth is that I went on a post-NIWeek 6-week vacation and am finally back at work. [Or perhaps Nancy was doing the other part of her job… helping customers be successful through face-to-face interaction.–ed.]
In the last post, we looked at the typical use cases and benefits for Packed Project Libraries (PPLs). However, as the Field Architects have been working with customers, we ran into a few issues.
We are faced with a design challenge when working on large projects with multiple Packed Project Libraries (PPLs), specifically layers of libraries. We need to understand how PPLs link to other PPLs when a hierarchy of PPLs exist. Let’s look at an example…
Public Method.vi and Private Method.vi are members of the library A.lvlib. When A.lvlib is added to a project, these owned VIs appear as well. Below, we have 3 files: A.lvlib, Public Method.vi, and Private Method.vi. The project library has a link (or reference–technically, the library is an XML file that lists the VIs and their location) that establishes the connection to the methods:
Once we compile A.lvlib to A.lvlibp, the PPL, we now have one file that contains, and not just references, all of the Exported VIs (former Public Methods):
We may continue to add PPLs as needed to our application:
Dilemma of a Hierarchy of PPLs
Our problem arises when, as is usually the case, we develop a hierarchical organization of libraries or PPLs. Let’s assume that A.lvlibp calls B.lvlibp and both are independently called by Main.vi. One might argue that A calling B and Main calling B is bad design. This design consideration should probably be the topic of a future blog post. Regardless, this structure isn’t uncommon. If we assume that B is a PPL of analysis functions, it is reasonable to assume that it might be called by more than one other PPL. It could be called by a PPL that measures data as well a PPL that reads data from a file. So our use case is as follows:
Let’s examine the multi-step process of compiling B.lvlib into a PPL.
We begin by compiling A.lvlib into a PPL called A.lvlibp. B.lvlib calls functions in A.lvlibp and as such is linked to A.lvlibp. One might assume that I could compile B into a PPL such that it is linked to the original compiled A.lvlibp. Alas, that is not the case. B.lvlibp treats its dependent A.lvlibp as a source file and creates a copy at a new location. At the conclusion of this process, we now have two copies of A.lvlibp. One resides in the original location in which it was created and the other resides in a new location.
Having two copies of A.lvlibp on the hard disk is arguably problematic, though this is the intended behavior of the PPL build process. When distributing an application, the executable and its dependencies are distributed together. This happens because of limitations in LabVIEW’s ability to dynamically search for and link to PPLs.
Is the PPL Source or Compiled?
A.lvlibp is indeed compiled code. However, for the purpose of creating a second PPL that calls or is linked to A, it is treated as a “source file.” Note that the original A.lvlibp is not source code, rather a source file for the PPL. Therefore when the compiled B.lvlibp is built, A.lvlibp is treated as a support file and must be copied to a second destination. That is how we arrived at having two copies of A.lvlibp.
So Now What?
The developer of Main.vi is now in a quandary with multiple copies of library A residing on disk. Can Main.vi call the original A.lvlibp and B.lvlibp? No; that causes a conflict of filenames. B.lvlibp is linked to the second A.lvlibp. We would then have a conflict in the project with two separate instances of A.lvlibp:Public Method.vi residing in two locations. So the way we handle this conundrum is to treat the original A.lvlibp as it is intended. It is the “source file” for B. We must plan and determine the destination of the final A.lvlibp, defining a sensible file structure for the developer of Main.vi who will use both A & B. In the above illustration, all PPLs called by main, regardless of hierarchy, reside in one folder.
What Else?
In the above use case, each PPL has a specific responsibility. Many PPLs may be called by Main.vi. Each can be updated with a new version, independent of the other PPLs. If a bug is found in either A.lvlibp or B.lvlibp, only the affected PPL must be retested and redeployed. However, multiple copies of the PPL will exist and, as noted above, the software development process should be designed appropriately to account for this.
Another Design?
Another option is to organize the libraries first. Remember that .lvlib files can be nested. B.lvlib (pre-compiled) can be a member of A.lvlib. The final and single PPL can be a compiled wrapper around the entire hierarchy.
So in this use case, one person might own the development, test, and maintenance of Library A. Another developer may do the same for B. Then instead of creating two PPLs, the entire hierarchy is deployed as one PPL. The benefit is that only one PPL exists and we no longer have multiple copies.
The downside is that we now have one monolithic PPL. If the end user is only using one or two VIs out of a library of hundreds, then this is not an optimum solution, as the load time will be longer. However, if the user will be using most of the PPL, then the load time issue is unavoidable.
What Do You Think?
Several of us differ on our preferred organization of layered PPLs so we’d like your input. Additionally, what issues still remain for you as you leverage PPLs in LabVIEW 2011? What could be changed to make PPLs more useful for you in 2012?
I have come across some of the very same issues. We started using packed libraries for our TestStand development – the result can be wonderful much easier for TestStand developers to work with (few files, no need for LabVIEW development environment / vi.lib dependancies) etc. In fact, if people use common code modules in TestStand I would love to generally recommend this approach.
However, we then came to a particular project that needed some custom code on top of our common libraries (to facilitate timing, etc, we needed to make sure things happened together and added extra processing etc). The goal was to then have a packed library for this custom code. However, because of the whole copying packed libraries things got messy. TestStand would get very upset that it could not load two VIs from different paths with the same names, etc. Even more complicating was the fact that TestStand cannot unload lvlibps (that’s a whole other discussion)
In the end we went down the monolithic approach – custom code links to the lvlibs associated with the packed libraries so it technically is using a different version lead us to a cleaner result.
Shaun – so are you saying TestStand not yet ready to use packed libraries…
I would not say that – We have had a lot of luck with it and have continued our roll out very effectively. The benefits (no vi.lib dependancies, simpler distribution, minimizing the chances of ending up with “customized” code on actual test stations, etc) have definitely outweighed the shortcomings.
The only issue was that TestStand doesnt unload Packed Libraries. So if you don’t need to unload your code modules, you’re fine (If you need to load the same thing from multiple locations for different sequence files etc, the use of Projects can be a work around).
Also, if you call your modules from multiple depths in the hierarchy (like this posting) just make sure that you build a monolithic library (i.e. use the source lvlibs instead of lvlibps when building your higher level modules).
I had a serious disdain for packed project libraries until I found a good way to use them for a plug-in architecture that avoids some of the difficulties of plug-in architectures I’ve attempted in the past.
I also simply avoid the multiple-copies-on-disk issue you mention above by only allowing Main to depend on a top-level lvlibp which defines an interface to the others.
See my document describing this architecture here:
https://decibel.ni.com/content/docs/DOC-19176
Please comment, I thrive on community feedback.
Maybe I found a way to improve using nested PPL’s.
http://lavag.org/topic/18065-nested-packed-libraries/?p=108395
Another trick is to always call the libraries using references. That way, the called libs are not in the build and one does not have to be afraid of multiple copies.
Nancy,
Thank you for this post which is still quite relevant a few years later. It was very helpful for clearing up some lingering PPL workflow confusion!