To get the best performance out of reviewer, you can configure each module with specific review criteria that fits your needs. All modules except description support criteria.
Criteria serve as guardrails for the reviewer when creating reviews and filtering hallucinations.
Each review must match specified criteria.
Best practices for Review criteria
- Clear and concise criteria produce better results
- Avoid broad criteria like
find all bugs or do not hallucinate
- Make sure to specify what reviewer should do instead of what it shouldn’t
Criteria templating
You can use Go templating to specify language specific criteria. Available variables are:
{{ .Language }} - represents file extension in format .<ext> (.go, .py, …)
Example use:
pr_review:
modules:
bug_hunter:
enabled: true
criteria: |
{{ if eq .Language ".go" }}
- Look for requests with context without timeout
{{ else if .Language ".ts" }}
- Look for incorrect use of unknown type
{{ end }}
Default Review Criteria
Each module comes with predefined set of default criteria.
You can modify these by specifying a criteria key for each module in .callstack.yml.
Specifying criteria in .callstack.yml will override the default criteria.
pr_review:
modules:
bug_hunter:
enabled: true
criteria: |
- Look for use of mutex.Unlock() without defer statement
- Look for requests with context without timeout
...
Bug hunter
- Logic errors
- Analyze the code's logic flow and identify any flaws that will lead to incorrect results
- Look for off-by-one errors, incorrect boolean conditions, or misplaced statements
- Check for proper error handling and edge case considerations
- Check for proper null checks and boundary condition handling
- Focus only on the additions in the diff hunks when looking for bugs
- Look for issues that will lead to bugs or unexpected behavior
- Spot inefficient algorithms or data structures
- Identify unnecessary loops or redundant operations
Security
- Identify SQL injection vulnerability
- Check for hardcoded credentials and secrets
Code Suggestions
- Evaluate adherence to {{ .Language }} specific coding standards and best practices
- Check for consistent naming conventions and code organization
- Identify overly complex methods that could be refactored
- Identify these code smells:
- God Object
- Data Clumps
- Large Class - only check if the code has too many responsibilities
- Duplicated Code
- Lazy Class
- Speculative Generality
- Shotgun Surgery
- Alternative Classes with Different Interfaces
- Excessive Use of Literals
