Automated Engineering Reports with Claude Code

The Weekly Report Nobody Should Have to Write

Inevitably, every engineering leader needs to report up on the state of their teams. Whether it’s a weekly sync with their manager, a monthly report to leadership, or a quarterly review with business stakeholders, the need to aggregate data from multiple sources and present it coherently is a constant request from upper management.

I’ve tried a few different approaches over the years: spreadsheets, document summaries, templates for my managers to fill out. But none of them were particularly scalable or easy to maintain. Between the ever-changing requests from leadership and wanting to save my managers a few hours a week, I decided to automate the process.

While I successfully created a Lambda-based solution on prior teams to extract and ingest data into a central data warehouse, I wanted to explore a different approach that would give not just signal but also insights and recommendations at a deeper grain. Bonus points if it could be done without having to maintain a database or infrastructure.

So what was I actually trying to solve?

The premise was simple: I needed to understand the state of our teams at a glance. This meant gathering data from multiple sources and creating a coherent narrative from it. This included Jira for ticket status, Slack for communications, and GitHub for code activity.

Understanding this data at both the team level and from the lens of individual contributors was crucial. Additionally, generating insights and recommendations from this data would be a huge value add.

Thinking of this as a product, I used the following first principles to guide the implementation:

1. It should be portable across teams
2. It should be easily extendable
3. The output should be actionable and easy to understand

The solution (for now)

An agent skill was the perfect fit for this case and easily addressed all of the requirements. For context, an agent skill is a structured markdown directory that equips an AI assistant with custom instructions and tool workflows.

1. Portability: The skill is easily ported across managers and teams through its onboarding step, which allows any user to define the relevant data sources and parameters for their team. The resulting configuration file could then be persisted to source control and shared across team members.
2. Extendability: The skill can be easily extended through the addition of new data sources and parameters. With a quick chat with Claude, I was able to add new data sources in minutes. Because skills are defined entirely as version-controlled text files, iterating and sharing them across the organization is seamless.
3. Actionable Output: The output of the skill is an interactive HTML report that is easy to understand and share. The report includes a summary of the data collected, as well as detailed breakdowns of the data by team and individual contributor. More valuable, though, is the analysis done by the agent to highlight potential concerns or opportunities.

In Practice

Onboarding is straightforward. The skill walks through the configuration process, attempting to discover relevant repositories, boards, and channels on its own. Getting my first report took just two commands. First, onboarding a fictional “Photon” team:

/engineering-report onboard photon-team

This creates the base configuration file and resolves the relevant repositories, boards, and channels for the team.

After the onboarding process, the skill is ready to generate reports. The following command generates a 30-day report for the team.

/engineering-report create a 30-day report for photon-team

The result is a comprehensive report that provides insights into the team’s performance and areas for improvement. Below is a snapshot of the executive summary, the full report can be found here.

Some useful insights I got on my first 30-day run for my actual teams:

Noisy visual-regression signal. VR runner flagged “changes detected” on nearly every build to main throughout the period — worth a gut-check with whoever owns the config.

Noisy synthetic monitor. Six incidents of a synthetic monitor running and alerting on a false positive across the period.

Both of these were interesting finds that otherwise would have gone unnoticed. These are things that we feel as a team, but they are hard to quantify. This allowed us to identify and address these issues.

Takeaways

There are problems that Engineering Managers face daily that can be solved with AI that go beyond the realm of code generation and review. By creating an agent skill, I was able to automate a process that would have taken hours of manual work every week and turn it into a simple workflow that could be executed in minutes.

Now, this is by no means a production-ready skill, but it’s a scrappy prototype that proved the concept and value of automating this process.