Customer Success Story
Anycode Security – AI fixed 95% of code security vulnerabilities (CAST, SonarQube & Checkmarx) on auto-pilot (by itself) for Fintech Enterprise Customer
Problem
Delivering high-quality software is hard. Nobody will deny or argue this simple truth. It takes significant effort and a lot of time to write the code and make it work. Today, the pressure to deliver as many functionalities as quickly as possible is higher than ever. Engineering teams work crazy hours to meet this goal. Unfortunately, this leads to an increased chance of deploying security-vulnerable code and not following coding standards & best practices.
Our Enterprise Client, a leading fintech company ($27B+ market cap), had 25,927 code security vulnerabilities across their codebase (2MM+ lines of code) diagnosed by the code scanning tools they use internally: CAST (25,365), SonarQube (544), and Checkmarx (18).
The challenge they faced in fixing the issues was the dilemma of choosing between the two:
Our Enterprise Client, a leading fintech company ($27B+ market cap), had 25,927 code security vulnerabilities across their codebase (2MM+ lines of code) diagnosed by the code scanning tools they use internally: CAST (25,365), SonarQube (544), and Checkmarx (18).
The challenge they faced in fixing the issues was the dilemma of choosing between the two:
- Delivering new functionalities to their clients but taking shortcuts (not following security and code quality best practices).
- Delivering significantly fewer functionalities but following security and code quality best practices.
Anycode eliminated the dilemma. We helped focus 100% of our client’s engineering effort on delivering new functionalities while our auto-pilot AI ensured the code was secure and followed best practices.
How did Anycode help?
First, we defined the priorities for our client:
- Whatever Anycode’s solution is, it must be 100% secure.
- Focus on DX (Developer Experience) – engineers should be liberated from doing grunt work.
- Metric-based approach – improving code scans was the main success metric.
How did we address the priorities:
- On-premise solution – Anycode is deployed within the organization's AWS account (no data ever leaves it, and the LLM is trained and fine-tuned by the client’s data only within their AWS account – Anycode’s public LLM does not have any trace of the data).
- DX Duo: GitHub Co-pilot + Anycode – our product is designed to collaborate with Co-pilot. Co-pilot empowers developers to create new functionalities while Anycode “cleans up the dishes” on auto-pilot and ensures the code follows security, scalability, and stability standards.
- Code Scans Before & After – the success metric is clear and transparent.
What was the step-by-step process we followed?
Week 1:
- We integrated Anycode to receive provision infrastructure and access to the Git branch and CI tests.
- Ran code scans (CAST, SonarQube & Checkmarx) to measure the “Before” state.
- Installed Anycode.ai on the codebase and created a new branch for the execution.
Week 2:
- Ran Anycode Auto-pilot within a dedicated branch. It assessed over 2,000,000 lines of code and fixed almost 26,000 code issues in 14 days* (we estimated that it would take a developer 90 days working full-time to complete the same scope of tasks).
*It took Anycode 14 days to assess and fix the entire codebase twice. The rest of the time was dedicated to setting up Anycode and reviewing the code. - Ran CI Tests (automated tests) on the modified codebase.
- Re-ran Anycode Auto-pilot and CI Tests – the first fix had issues with the code quality, so after fine-tuning the model, we ran the auto-pilot once again.
Week 3:
- We had our developers review the final code to ensure it was production-ready.
- Deployed the Anycode branch.
Week 1
We integrated Anycode to receive provision infrastructure and access to the Git branch and CI tests.
Ran code scans (CAST, SonarQube & Checkmarx) to measure the “Before” state.
Installed Anycode.ai on the codebase and created a new branch for the execution.
Week 2
Ran Anycode Auto-pilot within a dedicated branch. It assessed over 2,000,000 lines of code and fixed almost 26,000 code issues in 14 days* (we estimated that it would take a developer 90 days working full-time to complete the same scope of tasks).
*It took Anycode 14 days to assess and fix the entire codebase twice. The rest of the time was dedicated to setting up Anycode and reviewing the code.
*It took Anycode 14 days to assess and fix the entire codebase twice. The rest of the time was dedicated to setting up Anycode and reviewing the code.
Ran CI Tests (automated tests) on the modified codebase.
Re-ran Anycode Auto-pilot and CI Tests – the first fix had issues with the code quality, so after fine-tuning the model, we ran the auto-pilot once again.
Weeks 3-4
We had our developers review the final code to ensure it was production-ready.
Deployed the Anycode branch.
What challenges did Anycode face?
False Positives identified by Code Scanners (CAST, SonarQube, Checkmarx).
After the first Anycode Auto-pilot execution, we were concerned that it only fixed around 50% of all issues identified by Code Scanners, so our engineers had to manually review the issues Anycode Auto-pilot wouldn’t solve. To our surprise, almost all of those turned out to be correct code snippets with no issues.
How is this possible? The answer is simple – Code Scanners use complex algorithms to find patterns in code that they identify as issues, and sometimes it’s nearly impossible to distinguish the correct code from the one with issues.
In fact, Anycode Auto-pilot fixed 95% of all code issues.
So, the use case of Anycode is not only fixing the issues but also helping Code Scanning tools identify them more correctly.
Anycode struggles with large-scale changes to Code Structure.
Yes, Anycode Auto-pilot is not perfect yet, but it will get there very soon. Our team is currently working on making updates to the Auto-pilot to give it the ability to rewrite the entire codebase and ensure no business logic is corrupted.
What is the Impact of using Anycode? The project turned out to be more successful than the team and our client expected – Anycode solved 95% of code issues and liberated the client’s developers from doing the grunt work everyone hates while helping the team focus on what they truly love – innovating.
After the first Anycode Auto-pilot execution, we were concerned that it only fixed around 50% of all issues identified by Code Scanners, so our engineers had to manually review the issues Anycode Auto-pilot wouldn’t solve. To our surprise, almost all of those turned out to be correct code snippets with no issues.
How is this possible? The answer is simple – Code Scanners use complex algorithms to find patterns in code that they identify as issues, and sometimes it’s nearly impossible to distinguish the correct code from the one with issues.
In fact, Anycode Auto-pilot fixed 95% of all code issues.
So, the use case of Anycode is not only fixing the issues but also helping Code Scanning tools identify them more correctly.
Anycode struggles with large-scale changes to Code Structure.
Yes, Anycode Auto-pilot is not perfect yet, but it will get there very soon. Our team is currently working on making updates to the Auto-pilot to give it the ability to rewrite the entire codebase and ensure no business logic is corrupted.
What is the Impact of using Anycode? The project turned out to be more successful than the team and our client expected – Anycode solved 95% of code issues and liberated the client’s developers from doing the grunt work everyone hates while helping the team focus on what they truly love – innovating.
What is the Impact of using Anycode?
The Project turned out to be more successful than the team & our client expected – Anycode solved 95% of code issues and liberated the client’s developers from doing the grunt work everyone hates while helping the team focus on what they truly love – innovating.
