Make Libraries More

Software Libraries are good. They allow abstraction and encapsulation; which encourages reuse. They also allow the library to be written by one person and used by another. This is reuse at the programmer level, not just the system level. A library can be written by a person who has domain expertise but used by someone else who has less or no expertise in that domain. For example: an XML parser. The implementer knows the XML spec inside and out, but the user of the lib needs only a basic understanding of XML in order to use the lib.

Now that we've established that libraries are good (regardless of whether are are implemented as classes, packages, DLLs, etc.), how can we extend them? Traditionally libraries are very limited in how they can influence the experience of the programmer. The coder simply has new functions to call. A more advanced case is component driven GUI builders where the lib exposes more information so that the IDE can make the library easier to use. Annotating of types in a map component could enhance the GUI builder draw markers, use a dummy tile set, and set positions. This is all standard stuff. Could libraries do more? How could the library author further influence the experience of the library user?

Let's do a quick brainstorm:

• Extend the language with new operators and syntax specific to the library. A physics library could import operator overloads to enable more math like syntax when dealing with physical units and vectors.
• Tests that verify aspects of the code that calls the library. This could automate the advice that would be in the documentation, such as "don't pass null to certain functions" or "this function returns an object that you must dispose of".
• If the lib is used from within an IDE, the lib can insert new documentation into the searchable index of docs.
• The lib could come with example code and reusable annotated code snippets that would only be applied to projects using the lib.
• The lib could insert a new package manager into the compiler toolchain. No more messing with Maven POMs.
• The lib can provide the compiler with new code optimizers for the particular tasks. An imaging analysis library could instruct the compiler on how to better use SIMD instructions when available.

Obviously modifying the environment of the programmer is a dangerous activity so we would need very strict scoping on the changes. A library which extends the language applies only in the parts of the code that actually import the library. Changes are scoped to the code which asks for them.

Fundamentally a library is a thing created by one programmer for use by another programmer (or the same programmer at a future date). We should extend the concept of a library to apply to the entire programming experience, not just adding objects and methods. This would allow all environment extensions to be managed in a standard way that is more understandable, flexible, and scoped. I would never need to set up a makefile again. I could simply import in my main.foo code the package manager, repos, code chunks and syntax that I want to use for this project. The library extensions to the compiler handle everything else automatically.

Clearly there are challenges with such an approach. We would have to define all of the various ways a library could modify your environment. How would compiler hooks actually work? Annotation processors? A function to modify the AST a different stages of compilation? Would it imply a particular back end like LLVM? I don't have the answers. There clearly are challenges to be solved, but I think the approach is a step forward from today where we have many different components of our programming environment that are dependent on each other but don't actually know about each other. Having a way to organize this mess, even if imperfect, would certainly help us make better software.