An Interview with Meinte Boersma, author of Domain-Specific Languages Made Easy
Take 40% off Domain-Specific Languages Made Easy by entering intboersma into the discount code box at checkout at manning.com.
Meinte Boersma is a language engineer. As a long-time software developer, especially of language-based tools, he is driven to unlock the potential of Domain-Specific Languages (DSLs) to improve software development practices. Most recently, he has been working for the Dutch Tax Administration, using DSLs to put tax law “into the computer”—until June 2021, when the Dutch Health Ministry tapped him to work on an urgent Covid response program in coordination with the European Union.
On June 17th, 2021, Meinte Boersma sat down with Heidi Nobles, his development editor at Manning Publications, to discuss the Covid response project and how this high-stakes project highlights the value of DSLs for today’s developers. Meinte was in Voorhout, Netherlands; Heidi was in Charlottesville, VA, United States; they conducted this interview via Zoom.
The transcript has been edited and condensed for clarity.
Heidi Nobles: Hey, Meinte. So, for readers who may be unfamiliar, can you explain what a DSL is and why a group might need to create one?
Meinte Boersma: A DSL –domain-specific language—is a software language that is specific for a domain. But that says almost nothing, so I’ll unpack it a little bit.
A software language is a language that you can “speak” to a computer with to get it to do things. For the domain-specific part, we have to define what a domain is. If you go to Wikipedia you’ll see something like, it’s “a particular focus area of knowledge.” That’s probably not helpful. I tend to look for people working together on achieving a common goal. For example, we could think of a group of experts in a certain company, who together have to achieve something that involves a lot of business knowledge and business expertise, or other kinds of expertise. This group of people defines the “domain.”
And you can make these people more efficient, more effective, and more productive by giving them a DSL—a true computer language, created specifically for them to achieve their goal in a way that’s more efficient, more productive, more empowering than what they were doing before. And the funny thing is, most of the time if you have a group of domain experts, they already have their own language in the way they talk with each other, have a common set of concepts they work with and they all share a common understanding of. So you can see that natural domain language as a DSL that is not yet formally defined, and by making a tool to capture that domain language, we can bring together the work they’re doing both off and online.
Heidi Nobles: So in your book, you use the example of a rental car agency to explain a domain and how a DSL could help the team manage their workflow, especially managing complicated and dynamic rules over time. When you and I talk in editorial meetings, I tend to go back to my own primary domain—I’m a college professor, so I think of these complicated systems we have to manage students’ advising and enrollment, where we have hundreds or thousands of rules managing what classes students have to take, in what order, with what counterparts, in order to achieve certain milestones, and then the rules change every year, so the architecture of the computer system is a bit of a nightmare to manage. But a DSL can allow advisors and administrators—“domain experts”—to manage those rule changes themselves, without requiring the development team to handle constant tedious changes. Is that right?
Meinte Boersma: Yeah; that might be a helpful way for readers to think about it.
Heidi Nobles: So maybe let’s talk a little bit more about your current role, then you can go back in and apply that to the DSL model and that might help other people understand it a little bit better? Can you tell us a little bit about your current role with the Dutch Covid response team?
Meinte Boersma: Let’s start from the beginning. There’s Covid in the world—as we know—and that restricts travel quite a bit. That’s a problem for the European Union, because one of the mainstays of the EU is free travel among the 27 member states. During the height of the epidemic, there was no traveling. But now that we’re getting vaccinated, you want to allow that free travel again, not only because it’s a right of EU citizens but also because of things like tourism, as travel is a great boost to get economies going again.
So the problem is: how do you know that a certain someone, traveling from country A to country B, is safe to entry country B? How can someone at border control, or someone at an airport determine reliably that this person has been fully vaccinated (with the correct number of injections and a long enough waiting period dependent on the type of vaccine they received) or that they’ve had the virus within the right time period (long ago enough to have recovered but recently enough to have temporary immunity). So how does the border agent determine each person’s status?
What the EU has done is to come up with a thing called the Digital Covid Certificate, or DCC; it’s a certificate given out by the Health Institute in your own country (something like the Department of Health in the United States). So when you get your vaccination or you see your doctor after having recovered from Covid, your certificate will be updated, and then when you travel to a country you have to show your certificate, which is a QR code either on your mobile or printed out.
And then there is a verifier app that’s used by the border agent or someone at the airport or train station, wherever you have to travel through. They scan the QR code, and then the verifier app will say, well this person is totally fine, no problem whatsoever, or oops, no, this person cannot travel across this border.
In terms of the advantage of a DSL, this is where business rules come in, because this is really about making a decision based on shifting information and regulations. So in Britain, a traveler might be coming off a plane and the agent has to know, Okay, can we let them through, or does something else need to happen? Do they have to go into quarantine, or do you have to quick test them, or do we have to turn them back?
Ideally, you as the traveler want to be able to determine beforehand whether you will get accepted into a country, because it’s a bit tedious to plan and it’s certainly a bummer if you travel somewhere and then they have to turn you away. So you want to know the current rules as they apply to your travel situation.
Each EU member state can determine for themselves what their precise rules are; they can deviate from the common standard.
So, on the developer side, how do you write down these kinds of rules, in a way that you can interchange them among all the EU member states?
The solution we have chosen for that is to use a very small domain-specific language to express these rules in it. We’re building off of an existing language, called JsonLogic, which means that you send these rules around in some kind of JSON format. And that’s also a good idea because then you can actually download and upload these things and you don’t have to update entire apps in order to make new rules applicable. So this DSL is basically embedded in a JSON format.
I basically restricted JsonLogic to a very small subset, which is much, much easier, much less tedious to write good tests for. And now we’re in the process of writing down all the business rules for all the member states who will be using the DSL.
Heidi Nobles: Okay, now we’re a little over my head, so I’ll ask questions and if they don’t make sense you can change them as needed. So JsonLogic isn’t a language that you’re talking about in the book; you use other languages there. I guess I’m wondering, how common is it to kind append a DSL onto an existing language versus creating something completely from nothing?
Meinte Boersma: It’s quite common. Usually it’s called an internal DSL, but then you really piggyback the DSL on top of a programming language. For example, Ruby has a famous example with Ruby on Rails. Rails is basically a framework or a DSL on top of Ruby to do web applications.
My current case is a little bit different, because JsonLogic is not a programming language; it’s just a data interchange format. So this is a little bit special. I wouldn’t call it an internal DSL; I would call it a DSL for which is a serialization format and the notation coincides. So you’re basically using something that you normally use to serialize data as the format for your domain-specific language. That’s a bit more uncommon than an internal DSL where you really piggyback on top of a programming language that’s a lot richer in terms of syntax than a serialization format like JSON is.
Heidi Nobles: Okay, so let’s back up a little bit. Can you explain to readers how you got involved with this project? Because you were working with the Dutch government in your day job, and you got kind of commandeered into this team. So can you tell us a little bit about the team, how you got commandeered, and then what the team is like, in terms of who you’re working with and how it’s working?
Meinte Boersma: I got contacted by one of the team members who was already working on this project; a very nice fellow from the UK already living in the Netherlands for quite some time so he speaks Dutch perfectly. He has been following me on Twitter for more than a decade, I think. So when they figured out that we need someone who knows about business rules and language engineering, he thought of me and contacted me, and the rest is history.
So, that’s also nice example of sometimes the long game pays off. I’ve been writing blogs and tweets since 2009, or maybe a little bit earlier, and this is this is one of the occasions where it’s very clear that that has paid off in the sense that that effort builds your network and creates opportunities.
Heidi Nobles: So, the project that you’re on—Is there a name for the project that you’re on? I’ve just been calling it the Crisis Response Team or something but…
Meinte Boersma: I think the most commonly used name is the Corona team.
Heidi Nobles: Okay. So when did this team assemble and who’s in charge, or where’s that centralized?
Meinte Boersma: In the Netherlands, our team is ultimately led by the CIO of the Ministry of Health here. The team has existed I think since March 2020. And I’ve been involved since June 2021.
Heidi Nobles: Do you happen to know what they were trying before they switched to the DSL approach?
Meinte Boersma: They hard-coded it, which is fine, if you have to do with just for one country, and your scope is just the one country. But the thing now is that you want to exchange all these rules across all the member states.
Heidi Nobles: Yeah, this is such a helpful real-world example in terms of explaining the value of your book, for showing a time when you would use a DSL.
Meinte Boersma: Yes, because before, everyone was having to jump through hoops to get all these different apps for different countries, which may or may not understand your certificates and interpret them the same way as your apps do.
Heidi Nobles: And so on the (I think I’m using this term right?), on the backend, by making everything work through a DSL, you’re allowing everyone who would interact with this thing to make changes versus having to funnel it through one tiny group of people who would have to be doing the coding?
Meinte Boersma: Oh that’s also funny, because it’s not really backend work; it’s [about] verifier apps. They have to work without a backend. The data in a Digital Covid Certificate, is private data, and per EU regulations, it is not allowed to be stored anywhere. So the app really has to work in such a way that someone can present a certificate and this app can verify the validity of the certificate without consulting any background whatsoever.
Heidi Nobles: So the user data is all stored on their local device.
Meinte Boersma: Yep.
Heidi Nobles: And what about the business rules themselves?
Meinte Boersma: Those are also stored locally in this verifier app. Let’s say I want to travel to Finland. I have a look at my wallet app holding my certificate. The business rules are downloaded from a central gateway, but those are just the rules. They’re in JSON format, so it’s no problem to download those; the app does that for you. The app has the latest rules, and can run those rules against your certificate.
Heidi Nobles: So let’s say I’m a policy expert in Finland—not a programmer or developer—and I say, we’re going to change a policy: I now want to make sure that people are four weeks out (not just two) from their last vaccination before they enter the country. And I want to change that tomorrow. What does that look like?
Meinte Boersma: So you go through your DSL editor—
Heidi Nobles: —and just to be clear, the DSL editor is not a person, it’s an interface, right?—
Meinte Boersma: Yeah. So you open the interface for authoring these rules, which can ultimately be a JSON editor. You first have to say whether a certain existing rule has an end date. If something changes, it usually invalidates an existing rule, so you have to set the invalidation date for that rule. And then you make a new rule, like four weeks instead of two weeks.
You put in that new rule. And then you run some tests on it; you input some hypothetical test use cases to make sure the rules are working the way you want them to. You should put in a lot of test cases to check that your rule actually makes sense in real-life scenarios.
And after you verify that things are working as you want them to, you upload the new rule to the gateway. And then every time a verifier app connects to the Internet or is refreshed, then the app will ask the gateway, hey are there new rules? And the gateway will say, oh yeah, we have new rules for you. The app will download these new rules, and they will be active from the date you’ve selected.
Heidi Nobles: But as the policy person, I can pull it up on my editor, and I can modify those rule in a natural language that makes sense to me. And then I will have access to that testing component? Again, as a professor I’m thinking of how in my learning management system I can select “student view,” or I guess like on Facebook where you have the “View your profile as so-and-so” button?
Meinte Boersma: Yeah. And that segues into the projectional DSL question, because I intend to make a projectional DSL for the current project. So as I said before, the thing with this particular DSL is that the serialization format is JSON, which is currently also used as the notation. With a projectional DSL, that can be different, because a projectional DSL makes a very clear distinction between the serialization of the language data, and the way it’s presented to the user of the DSL—that is, toward the person authoring rules in the DSL.
For example, for traditional programming languages, the serialization format—how you store it—is a file of characters that is the same as the notation. When you’re programming, you usually have a code editor, which loads in this string of characters, and then you type in individual characters, and something has to make sense of those characters. But a projectional DSL works the other way around. You start with your DSL prose, your DSL utterances as data, and then you project that into a notation. And that’s called a projection.
Heidi Nobles: Since it’s come up multiple times on our end, of people being confused between projectional versus textual or graphical DSL—I think the textual DSL is easier for people to understand and recognize, oh, that’s not what he’s talking about. So maybe you could explain that difference briefly and then explain why projectional and graphical are also not the same thing?
Meinte Boersma: Yeah, let’s clarify terms a little bit. DSLs as we know them are usually textual DSLs. It’s a bunch of text characters in a file, which can be given some meaning by parsing the file. You have a separate program, like a compiler, that looks at the text of the DSL and then extracts data from that text. The text is quite low-level, and if you have a programming language that has, for example, a class definition or function, you have to work a bit before the compiler understands. So there is a step between going from the text in the file to a data structure that the compiler can work with.
With projectional DSLs, you’re skipping that step because you’re not saving individual characters in the file; you’re just storing the data that the compiler can work with directly. And you’re also not presenting your DSL prose, your DSL utterances, as low-level characters, but you’re presenting them directly from structured data that represents your DSL prose.
So skipping a step here actually makes things a lot easier. It also has a downside—you cannot just open a file and just start typing anymore; you need a specific editor for this DSL. But the upshot is that you can make this editor a lot more domain-specific than you can with textual DSLs. Because textual DSLs, again, are just characters in a file—so you can edit them with any code editor or any text editor, but your DSL will also feel like a text editor, all the time. And a textual DSL can be tricky to make: you have to make it specific for your domain; you really have to hack around in your text editor to give it more than just the text editor feel.
In a projectional DSL, you can make things really domain-specific from the get-go—you can give it specialized notation, you can make it graphical, you can it look like something more familiar (like a Word document). You can also switch the notation; you can have a switch somewhere allowing users to say, okay, I would like to see this in a table format instead of in a list format, things like that. So you have a lot of possibilities to make the projectional DSL very domain-specific.
Heidi Nobles: I think that answers the graphical question as well but is there anything else you need to clarify about that distinction?
Meinte Boersma: Yeah, there is a sort of thinking that, as soon as you’re talking about projectional DSLs, you’re automatically talking about graphical DSLs, and that’s not the case. Graphical is just one way of presenting DSL prose, and sometimes it works very well.
A typical case is data models. People think about data models in terms of what I call boxology—visualizing boxes and lines between boxes, where the boxes are entities and the lines are relations and references between those entities. Graphical works very, very well there. But for all kinds of other stuff, like expressions, formulas—those are usually very difficult to express graphically.
People are used to entering formulas, for example in Excel, as something that resembles text: You type 1+2. You don’t really care that this “+” is actually something that is binding this 1 and 2 together; you just think about them as 1+2. So for formulas, textual is typically better than graphical notation.
Heidi Nobles: Hmm. So—I’m trying to figure out why people keep having so much trouble with that…. And it occurs to me that when I see the term “graphical DSL,” it’s obvious to me that it’s referring to a DSL rendered as graphics. I think I try to extend that same intuitive/syntactic relationship onto the phrase “projectional DSL,” which means I end up telling myself, Okay, so it must be that the DSL is being rendered as a projection. But that’s not right….
Meinte Boersma: A projectional DSL is a DSL where your prose is always projected. There is some kind of storage format, which you’re mostly not interested in, and this stored data is then always projected into a nice notation for your DSL prose authors.
Heidi Nobles: Through the DSL editor.
Meinte Boersma: Through the DSL editor. You can also do it as a read-only thing, but for simplicity, let’s say the DSL editor.
Heidi Nobles: Okay, that’s helpful. So what might programmers find exciting about the language engineering that you’re doing now with the Covid response project?
Meinte Boersma: [laughs] Well, I’m not sure programmers would find it interesting, because they are cut out of the loop, so to speak.
Heidi Nobles: [laughs] Okay, yes. So what would your readers, your target readers of the book find interesting?
Meinte Boersma: What makes this interesting for developers is that they don’t have to care as much about programming these rules by hand and making test cases for them, because the rule makers will do that.
Heidi Nobles: And that has to be a tedious painful thing that they would like to avoid, yeah?
Meinte Boersma: Yeah, especially when you actually implement these rules, not only for a lot of member states, but also for a couple of technologies at the same time.
For example, when we say “app,” is it an iOS app, is it an Android app, or is it a website for people who are working from laptops? It’s usually a lot of work not only to implement all these rules for all the member states, but also to work across multiple technologies and keep everything in sync and test everything.
What’s interesting in this case is that we really start from a detailed description of these rules, from which we generate all kinds of stuff. For example, if we’re defining test cases along with the rules themselves, we can also generate QR codes to automatically test the apps.
So there’s a lot of work that you would normally have to do if you hard-coded the implementation of these business rules in these technologies, but you can just generate them and test across these various implementations and various technologies.
Heidi Nobles: You did say that the code for this project is open source, is that right?
Meinte Boersma: Correct.
Heidi Nobles: Is there a place that people can go find that?
Meinte Boersma: Yes; this is a link to the repository of “my stock”: https://github.com/ehn-dcc-development/dgc-business-rules
Heidi Nobles: In these last few minutes we’ve got, I wanted to ask you: What is it that you’re going to work on today?
Meinte Boersma: I have a couple of primary tasks.
First, we’re in the phase of the project where we’re trying to get all the member states to write their business rules for the DSL, instead of hard-coding them into the various programming languages they happen to be using. We just made a new repository today, where member states will be able to drop in their rule sets.
The other thing I’m working on today is projecting this JSON format into something more readable, because while these expressions are not super complicated, JSON is still quite wordy. It’s not XML, but pretty soon you have one screen full of JSON, just to express one business rule.
So right now, I’m making a projection for the JSON format that projects in a much more “human readable” form, and that’s just a read-only projection. Then slowly I’ll turn it into an editable prediction, so that users can author these things through a web app.
Heidi Nobles: Are you finding people on the various member state teams resistant? I mean, they put a lot of effort into the hard-coding, so are they resistant to transferring over, or do they see the benefits and they’re all on board?
Meinte Boersma: They’re not all on board yet. There’s also not really outright resistance. Generally, I explain this DSL approach, and they see the sensibility of it. What is special with this project is the deadlines and the timeliness, because basically every day is a sort of a deadline.
The gateway went live on July 1st, which is less than a month from when I started working on the team. And still a lot of work has to be done. So people are really in a mode of, we have to do something in such a way that we can have it done by tomorrow, and taking this approach is not really that. Most member states have had no other choice than just to hard code them up to now, which has meant putting off the problem of how we are going to coordinate all these rules with other members states to a later moment. Now we’re at that later moment.
So there’s no resistance, there’s just a mode of, this had to be done yesterday, so this is what we did. Give us something better and we will use it, as long as we can get it done tomorrow.
Heidi Nobles: Because you had essentially seven weeks from when you started to get this project done right?
Meinte Boersma: Yes, and that’s already an extended deadline, because the original deadline was June 1—which would have been 1 day before I started, which…. [laughter]
Heidi Nobles: [laughter] And all of the member states—they’re all hard-coding and they’re just doing what they can do on the fly, they’re all using different languages. They worked in isolation, because they had to, right?
Meinte Boersma: Yeah, and they’ve been free to create their own verifier apps, basically free to create every aspect of the apps and the IT landscape. So, sometimes that works. Sometimes they already have stuff done.
For example, a lot of the infrastructure in the Netherlands was implemented already a year ago, for various other Corona-related apps. It’s been natural to tag onto and use that. But other states didn’t have that much in place, so for them, it’s much more natural to look at the reference implementation that’s being made.
Heidi Nobles: So when this is done, will everybody get a new app that it just all goes through, and all the other apps will go away, or is this project going to sync with all the existing apps?
Meinte Boersma: I think it’s going to be, probably for most of the member states, a replacement of a certain part of their app—the part of the app that implements these business rules, which are now still hard-coded. That part is going to be replaced by this more generic engine that you can just feed JSON. This approach does the same thing as the original apps but is more flexible without needing to be hard-coded, and especially without needing to update your app as soon as rules change, which is very important when we’re managing the still-changing rules for 27 member states. In the old models, if one member state changed a rule, all the different apps would have to be updated—that’s not going to work. But with the new approach, the shared part of the apps should work in a couple of weeks.
Heidi Nobles: Is there anything else that you wish I would have asked you?
Meinte Boersma: I’d just want to let people know that I’m so busy with this project, the book-writing is on hold for a little while.
Heidi Nobles: I’m sure we understand that delay. But I’m so glad you were able to talk a bit in the midst of everything, and you HAVE written about ¾ of the book and it’s online if people want to begin reading. And if they buy now, they get that advance content and then they’ll get the whole book as soon as it’s published, which will be in a few months, as soon as you finish saving Europe.
Meinte Boersma: Yes; but first everyone needs to go on a holiday—which people in Europe can do because of this project.
Heidi Nobles: Okay, so they can spend some time on the beach, and then can come back to do what they really wanted to do, which is read about DSLs.
Meinte Boersma: Yeah. [laughter]
Heidi Nobles: What’s the elevator pitch about the book that you’ve been giving people?
Meinte Boersma: Hmm. If you want to empower your domain experts, and at the same time offload some tedium off of yourself as a developer, then you should seriously consider making a DSL for your domain experts, so that they can do their own programming. And then you can lie on the beach.
Heidi Nobles: That sounds good.
Well, thank you! I know this is a crazy hectic time, but I think this has been helpful.
Meinte Boersma: Thank you.
Meinte can be followed on Twitter @Meinte37
If you want to learn more about the book, you can check it out on Manning’s browser-based liveBook platform here.