Pull request analytics action

pull-request-analytics-action: A powerful tool for analyzing the effectiveness of both teams and individual developers. This action generates reports based on data from pull requests, code reviews, and comments, enabling you to identify your team's strengths as well as areas needing improvement. The statistics collected by this GitHub Action can be displayed in the form of tables and graphs or passed on for further operations as markdown or a data collection.

Table of Contents

• Key Features
• Getting started
• Configuration Examples
• Report examples
• Using GitHub Enterprise Server
• Detailed Report on Discussion Types
• Pull Request Sizes
• Report Data Grouping, AMOUNT Parameter, and Time Calculation Logic
• Configuration Parameters Overview
• Outputs
• Privacy and Data Handling
• Usage Limitations
• How You Can Help

Key Features:

• Customizable Tables and Graphs for Review Timelines: Generates user-friendly tables and graphs that mark critical milestones from PR opening to review, approval, and merge. Users can select the calculation method best suited for them, choosing from median, mean (average), or a selected percentile. This feature helps to identify bottlenecks in the code review process.

gantt
title Pull requests timeline(percentile75) 12/2023 / minutes
dateFormat X
axisFormat %s
section dev1
Time in draft(34 minutes) :  0, 34
Time to review request(17 minutes) :  0, 17
Time to review(3 hours 34 minutes) :  0, 214
Time to approve(7 hours 32 minutes) :  0, 452
Time to merge(14 hours 9 minutes) :  0, 849

section dev2
Time in draft(27 minutes) :  0, 21
Time to review request(12 minutes) :  0, 20
Time to review(4 hours) :  0, 240
Time to approve(4 hours) :  0, 240
Time to merge(23 hours 1 minute) :  0, 1381

section dev3
Time in draft(27 minutes) :  0, 15
Time to review request(12 minutes) :  0, 18
Time to review(15 hours 16 minutes) :  0, 916
Time to approve(24 hours 7 minutes) :  0, 1447
Time to merge(53 hours 43 minutes) :  0, 3223

section dev4
Time in draft(27 minutes) :  0, 13
Time to review request(12 minutes) :  0, 12
Time to review(2 hours 41 minutes) :  0, 161
Time to approve(9 hours 15 minutes) :  0, 555
Time to merge(47 hours 41 minutes) :  0, 2861

section dev5
Time in draft(27 minutes) :  0, 27
Time to review request(12 minutes) :  0, 18
Time to review(5 hours 59 minutes) :  0, 359
Time to approve(18 hours 55 minutes) :  0, 1135
Time to merge(40 hours 7 minutes) :  0, 2407

section total
Time in draft(27 minutes) :  0, 27
Time to review request(12 minutes) :  0, 18
Time to review(4 hours 21 minutes) :  0, 261
Time to approve(7 hours 36 minutes) :  0, 456
Time to merge(26 hours 14 minutes) :  0, 1574

pie
title Review time total 12/2023
"0-1 hours(12)":12
"4-6 hours(7)":7
"6-9 hours(4)":4
"12+ hours(2)":2

• Comprehensive Report on Merged PRs, Code Changes, and Reviews: This feature compiles a report detailing the number of merged PRs, lines of code modified, and reviews conducted. It provides an approximate measure of the workload, both for individual developers and the team as a whole, offering a clear view of productivity and contribution.

• Quality Report on devInitiated PRs: This feature generates a report analyzing the quality of PRs opened by developers. It collates data on the number of comments received, discussions held, and reasons for these discussions, along with the quantity of requested changes in open PRs, all presented in both tabular and graphical formats. This functionality aids in identifying the most problematic areas detected during code reviews and quantifying their extent.

pie
title Discussions types total 12/2023
"Bug(12)":12
"Performance(8)":8
"Code complexity(3)":3
"Test coverage(2)":2
"Formatting(9)":9

• Developer Engagement in Code Review Process: This feature assesses the level of developer participation in code reviews. It provides a table showing the discussions initiated, comments made, along with a breakdown of the number of code reviews conducted and the decisions made. This enables you to gauge the involvement of developers in the review process effectively.

• Highlighted PRs List by Key Metrics: One of the standout features of pull-request-analytics-action is the ability to generate a list of the most notable pull requests based on four key metrics: time from opening to review, time from review to approval, time from approval to merge, and the number of comments. This feature provides a list of links directly to these exceptional PRs, allowing for quick access and detailed analysis.

• Highly Customizable for Specific Project Needs: This action is designed with flexibility in mind, allowing for extensive customization of display parameters, statistics collection, and report generation. Users can tailor the tool to precisely fit the requirements of their specific projects, ensuring that the reports and analytics are as relevant and useful as possible.

• GitHub-Integrated, Secure, and Open Source: As a GitHub Action, pull-request-analytics-action operates entirely within the GitHub environment. It neither shares nor stores any data on external services, ensuring complete data security and privacy. Additionally, it is an open-source tool, providing full transparency in its operation, and is available to use at no cost, making it accessible for all GitHub users.

This GitHub Action, pull-request-analytics-action, is an essential tool for any team seeking to optimize their software development process, ensuring more efficient and effective project management.

Getting started

To integrate pull-request-analytics-action into your GitHub repository, use the following steps. The provided code is a template and can be adjusted to fit your specific requirements:

1. Create a Workflow File:

   • Navigate to the .github/workflows directory in your repository.
   • Create a YAML file, for example, pull-request-analytics.yml.

2. Choose the Trigger Event: Decide on which GitHub event you want to trigger the report generation. You can refer to the GitHub Events Documentation for a detailed understanding of different events. I recommend using one of the pull_request, workflow_dispatch, or schedule events, depending on your specific needs. For this setup, we will configure the workflow_dispatch event.

3. Insert and Customize the Workflow Code:

   • Open your new YAML file and paste the following example workflow. This is a starting template and you can modify it as needed:

     name: "PR Analytics"
     on:
       workflow_dispatch:
         inputs:
           report_date_start:
             description: "Report date start(d/MM/yyyy)"
             required: false
           report_date_end:
             description: "Report date end(d/MM/yyyy)"
             required: false
     jobs:
       create-report:
         name: "Create report"
         runs-on: ubuntu-latest
         steps:
           - name: "Run script for analytics"
             uses: AlexSim93/pull-request-analytics-action@v2
             with:
               GITHUB_TOKEN: ${{ secrets.TOKEN }}
               GITHUB_REPO_FOR_ISSUE: "repo"
               GITHUB_OWNER_FOR_ISSUE: "owner"
               GITHUB_OWNERS_REPOS: "owner-1/repo-1"
               CORE_HOURS_START: "9:00"
               CORE_HOURS_END: "19:00"
               TIMEZONE: "Europe/Berlin"
               REPORT_DATE_START: ${{ inputs.report_date_start }}
               REPORT_DATE_END: ${{ inputs.report_date_end }}
     

   • In the workflow_dispatch section of the yml file, I have specified various inputs that can be adjusted each time the action is triggered. By utilizing the required and default fields, I've designated whether each input is mandatory and set predetermined values for ease of use. In the with section, I've included parameters that remain constant for each action run. For a detailed understanding of which parameters the action accepts and their functions, please refer to the Parameters Overview section.

   • GitHub Token: The GITHUB_TOKEN is a special token used by the GitHub Actions runner to interact with your repositories. You can generate this token as per the GitHub Documentation on Authentication. When setting up the token, ensure you configure the necessary permissions. For pull-request-analytics-action to function correctly, personal access token(classic) requires repo and read:org scopes. Insert the token in your workflow file under the secrets context.

   • Adjust parameters to match your project's needs.

4. Commit and Push the Workflow File:

   • Save your changes and commit the file to your repository.
   • Push it to enable the GitHub Action workflow.

5. Run the Workflow:

   • In your repository, go to the 'Actions' tab.
   • Select PR analytics and start it via "Run workflow".
   • Fill in any necessary parameters and execute the action.

6. Review the Generated Report:

   • Once the action completes, your detailed PR report will be available.
   • If configured, check for a new issue in the specified repository containing the report.

This setup allows you to fully leverage pull-request-analytics-action for comprehensive PR analysis, tailored to your project’s needs.

Configuration Examples

1. Manual Execution for All Organization Repositories: Includes options to set start and end dates for the statistics period and to customize the report title. View config

2. Up-to-Date Statistics for Latest 200 Closed PRs: Automatically updates the report with every PR closure, maintaining current statistics for the last 200 updated closed PRs.View config

3. Yearly Statistics Update Every Monday: Configured to update yearly statistics every Monday, with assignees set and excluding individual developer statistics. View config

4. Quarterly Report Update on the 1st of Each Month: Updates the report for the last three months on the first day of each month, displaying only the average values for timelines and specifying assignees. View config

Report Examples

Explore how pull-request-analytics-action works with these report examples in the project:

1. Issue Creation with Comments: Demonstrates the generation of a new issue with detailed comments for each report. View Issue Example.

2. Multi-Month Report: Analysis over several months. View Markdown Example.

3. Last N PRs Report: Insights into the most recent pull requests. View Markdown Example.

4. JSON Data Report: Example of JSON data collected by the action, showcasing detailed analytics. View JSON Example.

Click the links for detailed report formats and insights.

Using GitHub Enterprise Server

pull-request-analytics-action supports integration with GitHub Enterprise Server. To use this feature, you need to set the GITHUB_API_URL environment variable:

1. In your workflow file, define the GITHUB_API_URL under the env key.
2. Set the value to your GitHub Enterprise Server API endpoint.

Example:

env:
  GITHUB_API_URL: http(s)://HOSTNAME/api/v3

This configuration allows pull-request-analytics-action to interface with your GitHub Enterprise instance, enabling you to leverage the full capabilities of the action within your enterprise environment.

Detailed Report on Discussion Types

To obtain a detailed report on the types of open discussions, it is necessary to include a specific label in the first message of each discussion, enclosed in double square brackets ([[ ]]). For example, use [[Performance issue]] to categorize a discussion as related to performance issues. The action will then provide a breakdown of discussions based on these labels, allowing for a more targeted and categorized analysis of discussion topics.

In addition, reactions of :+1: and :-1: on the initial message of a discussion will be collected and displayed in the columns for "discussions received" and "discussions conducted." These reactions help to more accurately gauge the usefulness and reception of the discussions initiated.

Example Usage:

• In the first comment of a pull request discussion, include a label like [[Bug]], [[Feature Request]], or any custom label of your choice.
• pull-request-analytics-action will recognize these labels and include them in the report, providing a categorized overview of discussions.

This feature enhances the analytical capabilities of pull-request-analytics-action, offering a deeper insight into the nature and distribution of discussions in your pull requests.

Pull Request Sizes

Calculating PR Size: The size of a pull request is determined using the formula: additions + deletions * 0.5. This calculation considers both the lines of code added and a weighted count of deletions to assess the overall size.

Based on this calculation, pull requests are categorized into the following size brackets:

• 0-50: xs
• 51-200: s
• 201-400: m
• 401-700: l
• 701+: xl

Report Data Grouping, AMOUNT Parameter, and Time Calculation Logic

• Data Grouping: The report data is organized based on the closure date of each pull request.

• Using AMOUNT Parameter: When AMOUNT is specified without REPORT_DATE_START, the report includes the specified number of most recently modified pull requests. However, the report count may be less than the AMOUNT specified, as it only includes merged pull requests.

• Excluding Weekends and Non-Working Hours: The calculations for the report exclude weekends. Furthermore, when CORE_HOURS_START and CORE_HOURS_END are set, time outside of these core working hours is not considered in the time-related metrics.

Configuration Parameters Overview

Below is a table outlining the various configuration parameters available for pull-request-analytics-action. These parameters allow you to customize the behavior of the action to fit your specific needs. Each parameter's name, description, requirement status, and default value (if applicable) are listed for your reference:

Use these parameters to tailor the pull-request-analytics-action to your project's specific requirements.

Outputs

Below is a table describing the possible outputs of pull-request-analytics-action:

Privacy and Data Handling

pull-request-analytics-action is designed with privacy and data security as paramount concerns. The action operates statelessly, meaning it neither stores nor transmits any data obtained during its execution. All operations are conducted entirely within the GitHub environment to ensure the utmost privacy and security of user data.

In version v2, to better understand user needs and to allocate resources more efficiently for the action's development, some non-sensitive input data are transmitted to Mixpanel. This is strictly limited to inputs that do not compromise the security or privacy of your projects. The complete list of inputs sent can be reviewed in this file.

Data transmission to Mixpanel can be disabled by setting the ALLOW_ANALYTICS input to false. However, we strongly encourage you not to disable this feature. The data sent poses no threat to your project's security or privacy but can significantly aid in the development and improvement of pull-request-analytics-action.

Usage Limitations

pull-request-analytics-action operates within GitHub's API rate limits and message size constraints, which are generally sufficient for detailed, long-term reporting. However, in rare cases of extremely large datasets, some adjustments might be necessary. For more information, refer to GitHub's documentation on rate limiting. The length of the report generated by pull-request-analytics-action is limited to 65,536 characters due to GitHub Issue size constraints.

How You Can Help

Contributions to pull-request-analytics-action are always welcome, no matter how large or small. Here are some ways you can help:

• Star the Project: If you find pull-request-analytics-action useful, consider giving it a star :star: on GitHub. This helps increase its visibility and shows support for our work.
• Spread the Word: Mention pull-request-analytics-action in your articles, blog posts, and social media. The more people know about it, the better it gets.
• Contribute to the Code: Follow our contribution guidelines to make code contributions. Every pull request helps!
• Report Bugs: Encountered an issue? Please let us know by opening an issue on GitHub. This is crucial for continuous improvement.
• Share Ideas: Have ideas on how to improve pull-request-analytics-action? Open an issue and tell us about your suggestions.
• Participate in Surveys: Occasionally, we conduct surveys to gather feedback. Your participation would be invaluable in shaping the future of pull-request-analytics-action.
• Get Featured: If your company or project uses pull-request-analytics-action, let us know! We'd be proud to mention your name in our list of users. It's a great way for you to showcase your commitment to quality in software development, and it helps us demonstrate the real-world effectiveness of our tool.

Your support and contributions greatly enhance this project. Together, we can make it the best tool for analyzing pull requests!