Developing a mathematical modelling language in my free time was no easy feat. After an insightful discussion with my professor, this self-directed project held an intricate nature and abundant opportunities for learning. Along the way, I encountered trials, errors, and countless insights that not only shaped the language but also my comprehension of the field.
Here is how I created the mathematical modelling language and the lessons I learned.
A glimpse into my background
I am an engineer with profound expertise in mathematical modelling and combinatorial optimisation. My academic journey culminated in a PhD in Industrial Engineering and a Master's degree in Computer and Electronic Engineering. Over the years, I've been deeply immersed in optimising route planning for solar-powered battery electric vehicles and addressing the challenges of passive optical network planning. My proficiency spans solver technologies such as Gurobi, CPLEX, and CLP/CBC. Moreover, I'm adept with programming languages like C++ and PHP.
But this isn't merely a tale of using tools; it’s about forging them. I created a mathematical modelling language (hlpl) that's versatile across different solver technologies and easy to adopt for those familiar with the typesetting language, LaTeX.
The Professor's Challenge: Bridging the gap between model and implementation
It began during an insightful conversation. Prof Fanie Terblanche, my PhD supervisor, once told me it's challenging to verify if a student's mathematical model and its implementation align.
He often reviews results from datasets provided by his postgraduate students, making it difficult to confirm the correctness of the implementation. While we typically use small datasets for some verification, one question still lingered: Is what you've implemented the same as what you wrote in your document?
Taking on the challenge, I embarked on creating a mathematical modelling language designed for optimisation problems, especially since that's Prof Terblanche's area of expertise. This modelling language would then be able to bridge the gap between written theory and computational implementation, ensuring a direct translation of mathematical models into executable code, reducing the room for discrepancies.
Starting out with LaTeX and my own lexer
I wanted to leverage the familiarity many researchers and students have with LaTeX's typesetting capabilities. By mirroring its syntax, I aimed to provide them with a tool that not only allowed for seamless mathematical modelling but also ensured consistent and professional typesetting in their documents. This allowed me to seamlessly extract and use the LaTeX code in documents.
In its infancy, my language was a basic text parser. I remember its limitations well - it inefficiently processed strings and lacked a structure for syntax. This experience sparked a deeper curiosity in me about the underlying mechanics of programming language syntax.
To enhance its functionality, I realised a better system for interpreting symbols was crucial. This realisation led to the development of my own lexer. With this addition, features like mathematical summations and the "for all" function became a reality in the language.
Although it still didn’t handle set-based operations, one could manually input data through structures resembling JSON. When I showcased this to my professor, he was pleased but posed immediate questions: "Is there an easier way to input data? And could I also do some data preparation?"
Incorporating Lua and C++ to improve functionality
Initially, it was just a fun project, but this prompted me to explore further. I had two options:
- Extend the language to include functions, which could be entertaining, or
- Why not use a simple language in conjunction with the modelling language?
I recalled that World of Warcraft used Lua to create plugins, so I thought I might employ a similar approach and extend the language with Lua. That's how Lua became part of the modelling language, and it worked great!
Surprisingly, I hadn't foreseen the potential of using scripting language functions within variable subscripts. It was Prof Fanie who pointed out this capability, broadening the scope and depth of what the language could accomplish.
Around this time, Prof Fanie shared some intriguing code with me that he had crafted during his leisure hours. This code, written in C++, served as an abstraction layer for modelling languages and had CPLEX and CLP/CBC neatly integrated. Realising the potential, I saw an opportunity to meld his work with mine, laying the foundation to make the modelling language more dynamic and functional.
Improving accessibility and usability through web development
Just when I believed I was grasping all the intricate aspects, a new challenge reared its head. Operating the modelling language on Linux, an operating system both Prof Fanie and I favoured, presented unique considerations.
Not everyone was familiar or comfortable with Linux, and the modelling language's early stages functioned as an unconventional command-line utility. This tool processed the model and data, attempting to decipher the optimisation problem described by the model. This phase highlighted a pressing need: enhancing accessibility and user-friendliness for a broader audience. This realisation further intensified my resolve to refine and evolve the language.
Given this newfound awareness, and with my penchant for diving into new tech territories, I ventured into web development.
The struggles of Haskell’s Snap framework
Back in 2018, I created a basic website harnessing Haskell's Snap framework. But, much like my previous endeavours, I soon discovered that theory and practice are two very different beasts.
The support was minimal, and the website's performance plummeted drastically upon running a model. The solution seemed straightforward: offload computations from the webserver. This did bring about an improvement— but only for individual users.
When a fourth-year and a PhD student tried it out, I quickly realised that while the system worked, it wasn't yet optimised for a streamlined user experience.
Collaboration led to the use of Laravel
In the midst of my web development journey with Haskell's Snap framework, an enthusiastic friend expressed an interest in joining the venture. However, the unique complexities of Haskell became a barrier for wider collaboration.
Assessing other tools, I contrasted the features of Spring Boot and Laravel. While both frameworks brought something valuable to the table, Laravel’s outstanding documentation won me over. Paired with the incorporation of a dedicated compute server, the project began to soar.
Originally tailored for my particular modelling language with its intense computational needs, I recognised its broader applicability. Why limit it? And with that revelation, the project's scope expanded, not only accommodating multiple mathematical modelling techniques but also integrating wider use cases beyond optimisation, such as simulation and data analysis, thus embracing its full potential.
Automating Deployment with Docker and Jenkins
After I expanded the project, I faced a unique challenge: the compute server needed to recognise its own capabilities and manage jobs from the 'service'. Manually maintaining the code for each server felt like a slog.
Enter my older brother, Juan. During one of our laid-back tech chats, he introduced me to a game-changing combination: Docker paired with Jenkins for automated deployment. And just like that, the process was streamlined.
Diversifying scripting languages and realising adaptability
The automated deployment strategy wasn't the only novelty I introduced, courtesy of family insights. Shortly after, Prof. Fanie approached me with another question: 'Can't I use Python with this?' This got the wheels turning.
The crux of the modelling language was invoking functions within the model that resembled pure mathematical functions—immutable inputs, deterministic output. This realisation prompted me to abstract the scripting language.
After some reshuffling and the birth of an abstract scripting language class, voila! We could incorporate any scripting language we fancied. It's worth pointing out that while Python and Lua might not inherently operate like pure mathematical entities, they can be tailored to play the part.
Real-world application and academic adoption
The adaptability of the modelling language didn't end with scripting languages. Its real-world applicability became more pronounced as I saw its adoption in academia.
At North-West University (NWU), it wasn't just a tool for study; it played a pivotal role in an advanced optimization course, with students even penning their practical exams using it. Its research application further cemented its credibility: two PhD students, one Master’s student, and a handful of final-year students turned to it for their projects.
Offering a free trial and always being on standby to assist with computational tasks for research, the language positioned itself not just as a utility, but as an academic ally
Lessons learned when embracing challenges
I learnt a lot through the challenge first laid out to me by Prof Fanie. It was quite a journey from conception to the final project - something that I don't regret. Here are some of the key lessons I've learnt that are good to keep in mind when embarking on your own challenge.
#1 Embrace failure
One of the most valuable lessons I've learned from this project is how instrumental failures can be in building in-depth knowledge. Every misstep or unexpected outcome in the development process offers you a chance to understand why something didn't work and how it could be improved. The insight you gain from these experiences is invaluable and will undoubtedly contribute to your growth and expertise in the field.
#2 Create to understand
Even if something already exists, building it yourself can provide a much deeper understanding. Reading about concepts or processes might give you a theoretical understanding, but creating something from scratch offers hands-on experience, bringing you face-to-face with the practical challenges and nuances that aren't always captured in books or lectures.
#3 Fail fast and learn
The faster you make mistakes, the quicker you learn. In a project as complex as building a modelling language, it's natural to encounter roadblocks and make mistakes. Instead of viewing these as setbacks, see them as opportunities for learning and improving.
#4 Learning beyond reading
Reading is undoubtedly an essential part of learning, but it's not the only way. Engaging in hands-on activities like coding, experimenting with different technologies, or even debugging can teach you much more. Building something tangible promotes a kind of learning that is experiential and practical, which is often more impactful and long-lasting.
#5 Collaboration is key
Seek insights from others, as their fresh perspectives can illuminate blind spots or offer novel solutions.
#6 Never shy away from challenges or be afraid of making mistakes
They are the cornerstones of learning and growth - so, venture forth, build, fail, learn and repeat!
Ruan Luies specialises in combinatorial optimisation with a PhD in Industrial Engineering. He's developed route planning for solar-powered vehicles and has expertise in tools like Gurobi, CPLEX, and programming in C++ and PHP. Ruan's passion lies in renewable energy and telecommunications solutions.
Read more
- How I Created my own AI Bot Game Using Lisp
- How I Used Web Dev Architecture to Structure my Home Automation System
- 4 developer-tested ways to get started with project-based learning
