How to Perform SonarQube Scans on Synapse Notebooks in Azure DevOps?

Question

I would like to run sonarqube scan on Synapse notebooks in Azure DevOps. In our Synapse notebooks, we have python, Pyspark, SQL and R codes. Our objective is to scan for vulnerabilities, code smells, bugs, etc.

We have write a code in DevOps for the same. Somehow, it is not addressing our objectives.

trigger: none
# - main
# - master

pool:
  vmImage: ubuntu-latest

steps:
  - checkout: self

  - script: |
      echo "Setting the working directory"
      cd $(Build.SourcesDirectory)/notebook
      echo "Current working directory: $(pwd)"
      ls -R
    displayName: 'Change Working Directory to notebook'

  - task: SonarCloudPrepare@3
    inputs:
      SonarQube: 'SonarCloud'
      organization: '****'
      scannerMode: 'cli'
      configMode: 'manual'
      cliProjectKey: 'ABCD'
      cliProjectName: 'ABCD'
      cliSources: 'notebook'  # Ensure this path is correct
      extraProperties: |
        sonar.sources=notebook
        sonar.inclusions=notebook/*.json
        sonar.verbose=true
        sonar.python.version=3.x  # Specify Python version
        sonar.language=py  # Only one language is allowed
        sonar.report.export.path=$(Build.ArtifactStagingDirectory)/sonar-report.html

  - script: |
      echo "Verifying notebook directory contents"
      ls -R $(Build.SourcesDirectory)/notebook
    displayName: 'Verify Working Directory'

  - task: SonarCloudAnalyze@3
    inputs:
      jdkversion: 'JAVA_HOME_17_X64'

  - script: |
      echo "Fetching SonarCloud analysis report..."
      curl -u $SONAR_TOKEN: -X GET "https://sonarcloud.io/api/issues/search?componentKeys=BPEODE_dssynapseprd" -o $(Build.ArtifactStagingDirectory)/sonar-results.json
      jq '.' $(Build.ArtifactStagingDirectory)/sonar-results.json > $(Build.ArtifactStagingDirectory)/sonar-report.html
    displayName: 'Export SonarCloud Results'

  - script: |
      echo "Listing files in ArtifactStagingDirectory..."
      ls -l $(Build.ArtifactStagingDirectory)/
    displayName: 'Debug - Verify Sonar Report File'

  # Step 1: Verify SonarCloud Report Exists
  - script: |
      if [ -f "$(Build.ArtifactStagingDirectory)/sonar-report.html" ]; then
        echo "##vso[task.setvariable variable=SONAR_REPORT_EXISTS]true"
      else
        echo "Sonar report not found!"
        echo "##vso[task.setvariable variable=SONAR_REPORT_EXISTS]false"
      fi
    displayName: 'Check Sonar Report File'

  # Step 2: Publish only if the report exists
  - task: PublishBuildArtifacts@1
    condition: and(succeeded(), eq(variables['SONAR_REPORT_EXISTS'], 'true'))
    inputs:
      pathToPublish: '$(Build.ArtifactStagingDirectory)/sonar-report.html'
      artifactName: 'SonarCloudHTMLReport'
      publishLocation: 'Container'
    displayName: 'Publish SonarCloud HTML Report'

Please help us to get the scanning report of the analysis.

Thanks in advance.

Answer 1

I can reproduce the same with your task. This is because you didn't define $SONAR_TOKEN properly in your yaml.

In Export SonarCloud Results, please fix the format of $SONAR_TOKEN as $(SONAR_TOKEN), so that the token can be invoked correctly.

The sonar-report.html in artifact contains the content correctly: