Building Tiny AI Tools Part 2: Evolving JiraGPT with Real User Feedback and Cursor Assistance

Addressing Early Challenges, Implementing Multi-Project Support, and Leveraging Cursor for Development

Intro

It’s been a week since I published how I built JiraGPT as the first tool in the Tiny AI Tools bundle. The reactions were great, especially from people curious about AI and automation. So, in the spirit of building in public, here is part 2. This time, we are still working on JiraGPT. Since last week, a few users started to use JiraGPT, or at least tried to, and that revealed some limitations, bugs, and a couple of missing features or things I didn’t account for. The big takeaway for me is you never know until real users start using it, no matter how much you test. So stay with me; I’ll walk you through how those limitations were discovered, how I tackled them, and how Cursor—as my new junior engineer—was a huge help!

User Doesn’t Have to Be a Jira Admin

When I was testing JiraGPT, I used my Jira admin account. So when the background job that syncs the project’s metadata kicked in, it never had any issues, and it shouldn’t have—except one of the Jira REST APIs we use is this one, which we use to fetch display names of users in a project. That API requires the user to be an admin. As you can imagine, this broke when a non-admin user installed the app; things started to break. Luckily, I found a quick workaround using another API: this one, and that was it. I noticed this issue because I had Sentry integrated, which is great for error monitoring. I can’t stress enough the importance of error monitoring even for an MVP/beta product, especially when it’s so easy to integrate Sentry.

What If a User Has Multiple Jira Projects? AKA Adding Multi-Project Support

Well, going back to my point about you never know until real users start using your app, in this case, they just connected Jira, but that was enough to stretch JiraGPT’s limits. The user works for a big and well-known company; they have lots of projects in their Jira instance, and in each project, they have hundreds of users. Up until this point, JiraGPT only supported a single Jira project, i.e., the user has one Jira project in their instance. Obviously, that is not ideal and had to change.

The challenge for me was how to do that in the easiest way possible without disrupting the user and with a clear user experience. After some whiteboarding and brainstorming with ChatGPT (o1-preview), I decided the flow should be as follows: when syncing the user’s project, check if they have a multi-project setup. If not, we shouldn’t disrupt those users; i.e., they don’t have to specify which project to search against in their queries. If they have multiple projects, the first time they ask a query without specifying a project, we prompt them to include which project they mean, and that’s only required for the first time. We then cache that so they don’t have to specify the project in subsequent queries. They still have the option to change that by specifying a different project in their query.

After testing that on my test projects and a real project, it seemed to work pretty well. Now, as a user with multiple projects, you can use JiraGPT.

Cursor Was a Huge Help

I switched from GitHub Copilot to Cursor lately (more on that in a different article), and it’s been great. But still, you kinda have to act as the architect/driver and, most importantly, a code reviewer; otherwise, you end up with jargon code that doesn’t work. So even though I’ve been advocating for the importance of having a good, detailed prompt when interacting with LLMs (ChatGPT, Claude, etc.), I’ve kinda been neglecting that advice with Cursor, partly because it’s been great. But when it comes to complex tasks, it hallucinates and over-engineers solutions. So this time, when I wanted to add multi-project support, I went back to my advice and wrote down a more detailed prompt highlighting what I needed. I even already provided it with the system design I had in mind and asked it to ask questions and confirm it understood what we want to build before diving into building it. As expected, it worked mostly from the first time with minor tweaks about UX. So here is the prompt (don’t mind the typos); hope it can be helpful to you:

Let's work on multi-project support. Currently, JiraGPT supports single project setup, i.e., the user has only one Jira project in their connection. We should modify the `JiraConnection` table to have that info and update it every time we sync the user's project. It should be a boolean flag. 

In the Slack integration, we should check if the Jira connection is multi-project. If not, keep doing what we have been doing so that users with a single Jira project are not disrupted. If, on the other hand, the connection is multi-project, we do the following:

a) If the user mentions a project in their query, use it to run the query as we don't support queries spanning multiple projects, and then cache the selected project (by cache, I mean save it on the `JiraConnection` table; we might cache it later in Redis), so that we can use it in subsequent queries. That way, the user doesn't have to specify it every time. It should only change when the user does so by specifying a different project in a query.

b) If the user doesn't mention it, we ask them to include it as part of their query just for the first time, and the rest of point a continues.

Notes:
1. Check `models.py`, `jira.py`, `slack.py`, `jql_builder.py`, `views.py` carefully and don't break existing features/code.
2. Make sure to implement it in a safe and secure way.
3. Solution should be simple.
4. Have a good user experience and clear copies.
5. If anything is not clear, ask me questions before diving into the implementation.

What Is Next for JiraGPT and TinyAI.Tools?

The next thing for JiraGPT is to talk to some customers and see where things go from there, in addition to fixing bugs and improvements.

The next thing for Tiny AI Tools is to work on the next AI tools. It’s probably gonna be for businesses; I have a few ideas, so hopefully I’ll pick one and build a quick MVP. As usual, I’ll share more details on how I’m building it, lessons, and challenges, so make sure to subscribe to the newsletter. I’d love to hear from you if you liked the article and your thoughts on things we’ve discussed—any AI ideas you think I should work on, tips, etc. Until next time, have a great one!