Fixing leaky laws

Normally we say a law is watertight when there are few or no loopholes, i.e. no reasonably practical ways of bypassing the law. Watertight laws without loopholes are desirable as they save us having to refactor / fix bugs in the laws later down the line.

Besides, it's usually the wealthy and/or other privileged parties who are able to take advantage of loopholes, resulting in an unequal application of the law across social demographics.

Thus, preventing or at the very least minimizing loopholes as much as possible is desirable.

In many modern democracies the law is usually formulated through multiple successive iterations of debate and discussion amongst people elected to represent the population, no matter their actual subject matter expertise. If loopholes are to be found in new laws it is down to the efforts within these iterations to do so. They can be effective in finding loopholes to an extent but if only if it's in the participating members' interests to do so.

For instance, if my election win was largely enabled by a wealthy oil industry donor then I might have a tendency to overlook loopholes in any new laws pertaining to the oil industry. And this isn't hypothetical. It happens quite regularly.

In software engineering there exists a concept known as automated testing. The idea is simple - as well as writing the code that runs your application, you also write separate code that, when executed, would test the application code to make sure that it behaves correctly and to required specifications. And this second set of code usually doesn't get be shipped with the final application since it's main purpose is for such internal testing.

Thus, you can test your code through multiple iterations prior to deployment. The testing can sometimes be semi-automated (i.e. some tests are run through by a human) - this is usually when the system has too many interconnecting parts to make automated testing possible.

Automated tests are great as they take less human effort to run, and thus get run more often. Usually a product has a suite of automated tests that get run after every code update to ensure that everything still works as it should. This makes it easier for teams of people to separately make changes to the application code, safe in the knowledge that as long as the automated tests are kept up-to-date and all pass that the application still works as it should. Automated tests written as code also act as a sort of natural documentation for what the application code does.

One of the classic - but not often followed - recommendations when writing automated tests is to ensure that the person writing the test code is not the same as the person writing the application code.

Why?

Programmers are generally lazy when it comes to writing tests and they are under pressure to deliver products to a deadline. Besides, not everyone adheres to high standards when it comes to testing their own code because they'd rather feel right than wrong. Having a different, dedicated person write the tests thus works better in terms of producing more rigorous tests and thus enhancing the quality of the overall application code.

Now that we know how automated testing works, we can apply this concept to the drafting of laws. What if we had a separate group of people whose job it was to "test" bills prior to them being published as laws?

It would be easier to do this if we write the tests as code. This means being able to write and represent laws as code. There are efforts in this direction. But it's likely not possible to express every idea in programming syntax. I'm thinking of freedom of speech laws as an example.

This is where AI can play a role. An AI that is trained on language as well as general knowledge of the subject area related to the law should be able to suggest test cases for a given law. Most importantly, it will let us do this in an automated fashion. In the long run I expect we will be able using AI to augment the law codification process itself to make laws easier to test and execute.

In the case of human testers (either writing and developing the tests or overseeing the AIs who do it), how can we trust them to do their job thoroughly? Will they be as susceptible to influence peddling by donors/lawmakers/lobbyists?

This could be approached in many ways but I'd like to try with a crypto economic perspective - "staking and slashing" - to see how it could play out. For example, for every iteration of a bill:

1. Lawmakers stake an amount of money of their choice when submitting a new bill for testing. They choose how long the testing will run for.
2. During testing, for every loophole testers find they slash a portion of that money and receive it as payment.
3. Any remaining money in the stake is returned to lawmakers once testing is done and is their bonus income on top of their base salary.

The iterations would thus look like:

Some key points:

• The loopholes found would need to be verified. Who will verify and what is their incentive to do so? This would need to be solved, though having the law represented as code makes this a lot easier, even to automate.

• The base salaries have to be quite low in order to incentivize participation, like a sales job mostly based on commissions would be.

• The money being staked by lawmakers come from the public pot and is there for them to earn as bonuses. Obviously there is a limit to how much they can be allowed to stake.

• By choosing how much they stake and how long testing will run for, lawmakers entice testers to choose their bill first over others. They must set a price low enough such that they won't lose too much in case of loopholes, and yet high enough to be meaningful to testers. The duration also cannot be too short, as testers will feel like there's not enough time. But it also can't be too long as that gives testers too much time to find more loopholes.

One interesting aspect of the above flow is that it might become possible to enable an individual lawmaker from drafting their own bill to send for testing. Of course, would still need the consenus of all lawmakers to pass the bill into law.

We want the number of available testers to be as large as possible to enable speedy testing and lower testing prices. Putting this in mind, there needs to be a permanently available group of people employed by the state whose job it is to test laws - this ensures that laws can always be tested. Additional random people would be able to join the "testing house" and bid for testing jobs.

As an aside, the House of Lords in the UK could instead become the "House of Testing" 😁

Note that it will always be possible to find a loophole given enough money, time and resources. The output of such a testing process would be based on the trade off that is deemed to be acceptable in terms of the law being watertight vs the amount of justifiable effort in making it so. The difference compared to what we have today is that by having a formalized testing process, we can clearly document the tests, loopholes, tradeoffs, etc for all to see. Let's call this "Open Law Drafting".