On Tuesday, July 27, our engineers deployed an update to permissions on our items API. This particular API call is is normally used to list products and services, and should only be accessible to users who have a role of Owner or Admin. However, after deployment, we received reports of problems adding job charges. After investigating, we found that the same API call used in the item list view was also being used to search for an item while adding new job charges. This meant that Staff and Tech users were no longer able to add a job charge without running into an error.

As usual, the remedy for this sort of bug involves two steps: 1) stop the immediate symptoms; and 2) address the underlying architectural issue. We took care of step one by putting permissions back the way they were, and job charges are now functioning properly again. For step two, we’ll need to make the API work the way it always should’ve worked: use a different API call when searching for an item to add as a job charge. This is a very different use case than showing the main list of items: one is an administrative function, and the other is used in the normal job workflow. They should be treated differently.

We’ll also be reviewing our internal system documentation to come up with a more reliable way of tracking what API calls are used in what places. The Kickserv codebase is large and constantly changing, and some parts of it are close to ten years old—which is a lot of lines of code for a developer to have to keep track of! We’re always looking for ways to improve quality of life for our engineers as well as our customers. We regret that this happened, and thanks for bearing with us.