Use Lucent Sky AVM with a continuous integration pipeline

This article describes how to integrate Lucent Sky AVM with a CI pipeline in CI tools such as CircleCI, GitLab, and Jenkins. By integrating Lucent Sky AVM with an application's continuous integration pipelines, developers can ensure that only code changes meeting the security standards are committed or deployed.

The Lucent Sky AVM CLI commands used in this article favor simplicity over scalability. For example, asynchronous methods such as BeginAnalyze might be more suitable than their synchronous counterparts when working with a large application. To learn about more advanced functionalities of the CLI, view the following article in the Lucent Sky Knowledge Base:
Lucent Sky AVM CLI reference

In this article, you will learn how to:

Prepare Lucent Sky AVM CLI for use in a CI pipeline.
Start a scan in a CI pipeline.
Download and evaluate a scan report in a CI pipeline.
Download the remediated source code and create a branch a CI pipeline.

At the end, you will be use Lucent Sky AVM in a CI pipeline to start a scan, download and evaluate reports, and work with remediated source code in a CI pipeline.

Prepare Lucent Sky AVM CLI for use in the CI pipeline

Lucent Sky AVM CLI needs to be downloaded to the CI server or agent before it can be used. How the CLI is downloaded varies and depends on each organization's needs and requirements, and the CLI might already be present if using a persistent agent.

In the CI pipeline, create the following variables:

ApplicationId - the value of the project's application ID.
SourcesDirectory - the directory where the application source code is located.
ToolsDirectory - the directory where Lucent Sky AVM CLI will be installed to.
ApiKey - a secret variable to store the API key to the Lucent Sky AVM server.

In the CI pipeline, locate an appropriate location to download the CLI, such as after the application build is completed.
Add a task (also known as step in some CI software) that runs a shell command to download the CLI setup file to $(System.DefaultWorkingDirectory)/tools/clear-cli.zip and extracts its content to $(System.DefaultWorkingDirectory)/tools/clear-cli.
```
 Invoke-WebRequest -Uri 'https://lsky.co/clearcli' -OutFile "$Env:ToolsDirectory/clear-cli.zip"
 Expand-Archive -Path "$Env:ToolsDirectory/clear-cli.zip" -DestinationPath "$Env:ToolsDirectory/tools/clear-cli"
```
The URL https://lsky.co/clearcli points to the latest Lucent Sky AVM CLI. To pin the CLI to a specific version, append -version after the URL. For example, https://lsky.co/clearcli-2309.

Add a task that runs a shell command to configure the CLI to use a remote CLEAR Engine host at clear.contoso.com.

 cd "$Env:ToolsDirectory/clear-cli"
 ./SkyAnalyzer.Interface.Console.exe --Interface config --Method set --Value "endpoint = clear.contoso.com:5759"

Start a scan in the CI pipeline

In the CI pipeline, locate an appropriate location to start the scan, such as after the build artifacts are available.
Add a task that runs a shell command to generate a random scan ID.
```
 $scanId = New-Guid
```
You may need to write $scanId to a specifc file or device in order to use the scan ID in later steps.

Add a task that runs a shell command to create a scan under the application on Lucent Sky AVM server and upload the build artifact for analysis.

 cd "$Env:ToolsDirectory/clear-cli"
 ./SkyAnalyzer.Interface.Console.exe --Key "$Env:ApiKey" --Interface Scan --Method Create --ApplicationId "$Env:ApplicationId" --ScanId "$Env:ScanId"
 ./SkyAnalyzer.Interface.Console.exe --Key "$Env:ApiKey" --Interface Scan --Method Analyze --ScanId "$Env:ScanId" --SourceCodePath "$Env:SourcesDirectory/target/ContosoWeb.war"

Add a task that runs a shell command to check the scan result code to determine if the scan was completed successfully.

 cd "$Env:ToolsDirectory/clear-cli"
 $scanResult = ./SkyAnalyzer.Interface.Console.exe --Key "$Env:ApiKey" --Interface Scan --Method GetResult --ScanId "$Env:ScanId"
 if ($scanResult -lt 0) { [Console]::Error.WriteLine("Scan failed ($scanResult)"); exit $scanResult }

You may need to configure this task to fail with either the presence of stderr or a negative exit code.

Download and evaluate a scan report in the CI pipeline

In the CI pipeline, locate an appropriate location to evaluate the scan report, such as after the scan is completed.

Add a task that runs a shell command to generate and download the XML report of the scan.

 cd "$Env:ToolsDirectory/clear-cli"
 ./SkyAnalyzer.Interface.Console.exe --Key "$Env:ApiKey" --Interface Scan --Method Report --ScanId "$Env:ScanId" --ReportPath "$Env:TempDirectory/ScanResults/Xml-Report.zip" --ReportFormat xml

Add a task that runs a shell command to extract the XML report.

 Expand-Archive -Path "$Env:TempDirectory/ScanResults/Xml-Report.zip" -DestinationPath "$Env:TempDirectory/ScanResults"

Add a task that runs a shell command to query the XML report to evaluate if the scan has found any vulnerability with a priority score of 2 or higher.

 cd "$Env:ToolsDirectory/clear-cli"
 $resultCount = ./SkyAnalyzer.Interface.Console.exe --Interface Query --Method Execute --QueryDataSource "$Env:TempDirectory/ScanResults/Report.xml" --QueryStatement "SELECT COUNT(ID) FROM Results WHERE PRIORITY <= 2"

You may need to write $resultCount to a specifc file or device in order to use the number of results in later steps.

Add a task that publishes the build artifact if no vulnerability with a priority score of 2 or higher was found.

Add a task that runs a shell command to generate and download the HTML report when at least one vulnerability with a priority score of 2 or higher was found.

 if ($Env.ResultCount -ne 0)
 {
   cd "$Env:ToolsDirectory/clear-cli"
   ./SkyAnalyzer.Interface.Console.exe --Key "$Env:ApiKey" --Interface Scan --Method Report --ScanId "$Env:ScanId" --ReportPath "$Env:TempDirectory/ScanResults/Html-Report.zip" --ReportFormat html
 }

Add a task that runs a shell command to extract the HTML report when at least one vulnerability with a priority score of 2 or higher was found.

 if ($Env.ResultCount -ne 0)
 {
   Expand-Archive -Path "$Env:TempDirectory/ScanResults/Xml-Report.zip" -DestinationPath "$Env:TempDirectory/ScanResults"
 }

Add a task that publishes the HTML report when at least one vulnerability with a priority score of 2 or higher was found.

Download the remediated source code and create a pull request in the CI pipeline

In the CI pipeline, locate an appropriate location to work with the remediated source code, such as after the pipeline failed security policy evaluation.

Add a task that runs a shell command to configure Git on the pipeline agent and creates a branch for the remediated source code when at least one vulnerability with a priority score of 2 or higher was found.

 if ($Env.ResultCount -ne 0)
 {
   cd "$Env:SourcesDirectory"
   git config --global user.email "[email protected]"
   git config --global user.name "CI Pipeline"
   git checkout -b scan-$Env:ScanId
 }

Add a task that runs a shell command to generate and download the remediated source code when at least one vulnerability with a priority score of 2 or higher was found.

 if ($Env.ResultCount -ne 0)
 {
   cd "$Env:ToolsDirectory/clear-cli"
   ./SkyAnalyzer.Interface.Console.exe --Key "$Env:ApiKey" --Interface Scan --Method Remediate --ScanId "$Env:ScanId" --RemediatedSourceCodePath "$Env:TempDirectory/ScanResults/RemediatedSource.zip" --RemediationOption 0
 }

Add a task that runs a shell command to extract the remediated source code over the original source code when at least one vulnerability with a priority score of 2 or higher was found.
```
 if ($Env.ResultCount -ne 0)
 {
   Expand-Archive -Path "$Env:TempDirectory/ScanResults/RemediatedSource.zip" -DestinationPath "$Env:SourcesDirectory"
 }
```

Add a task that runs a shell command to commit the remediated source code to a branch when at least one vulnerability with a priority score of 2 or higher was found.

 if ($Env.ResultCount -ne 0)
 {
   cd "$Env:SourcesDirectory"
   git add **/ApplicationProtectionLibrary.jar
   git commit -m "Instant Fixes from scan $Env:ScanId"
   git push -u origin scan-$Env:ScanId
 }

Add a task that runs a shell command to , which fails the pipeline when at least one vulnerability with a priority score of 2 or higher was found.
```
 if ($Env.ResultCount -ne 0)
 {
   [Console]::Error.WriteLine('This build did not pass the scan criteria.')
   exit -1
 }
```
You may need to configure this task to fail with either the presence of stderr or a negative exit code.
Create a pull request from the branch containing the remediated source code.