I would like to run sonarqube scan on Synapse notebooks in Azure DevOps. In our Synapse notebooks, we have python, Pyspark, SQL and R codes. Our objective is to scan for vulnerabilities, code smells, bugs, etc.

We have write a code in DevOps for the same. Somehow, it is not addressing our objectives.

trigger: none
# - main
# - master

pool:
  vmImage: ubuntu-latest

steps:
  - checkout: self

  - script: |
      echo "Setting the working directory"
      cd $(Build.SourcesDirectory)/notebook
      echo "Current working directory: $(pwd)"
      ls -R
    displayName: 'Change Working Directory to notebook'

  - task: SonarCloudPrepare@3
    inputs:
      SonarQube: 'SonarCloud'
      anization: '****'
      scannerMode: 'cli'
      configMode: 'manual'
      cliProjectKey: 'ABCD'
      cliProjectName: 'ABCD'
      cliSources: 'notebook'  # Ensure this path is correct
      extraProperties: |
        sonar.sources=notebook
        sonar.inclusions=notebook/*.json
        sonar.verbose=true
        sonar.python.version=3.x  # Specify Python version
        sonar.language=py  # Only one language is allowed
        sonar.report.export.path=$(Build.ArtifactStagingDirectory)/sonar-report.html

  - script: |
      echo "Verifying notebook directory contents"
      ls -R $(Build.SourcesDirectory)/notebook
    displayName: 'Verify Working Directory'

  - task: SonarCloudAnalyze@3
    inputs:
      jdkversion: 'JAVA_HOME_17_X64'

  - script: |
      echo "Fetching SonarCloud analysis report..."
      curl -u $SONAR_TOKEN: -X GET "; -o $(Build.ArtifactStagingDirectory)/sonar-results.json
      jq '.' $(Build.ArtifactStagingDirectory)/sonar-results.json > $(Build.ArtifactStagingDirectory)/sonar-report.html
    displayName: 'Export SonarCloud Results'

  - script: |
      echo "Listing files in ArtifactStagingDirectory..."
      ls -l $(Build.ArtifactStagingDirectory)/
    displayName: 'Debug - Verify Sonar Report File'

  # Step 1: Verify SonarCloud Report Exists
  - script: |
      if [ -f "$(Build.ArtifactStagingDirectory)/sonar-report.html" ]; then
        echo "##vso[task.setvariable variable=SONAR_REPORT_EXISTS]true"
      else
        echo "Sonar report not found!"
        echo "##vso[task.setvariable variable=SONAR_REPORT_EXISTS]false"
      fi
    displayName: 'Check Sonar Report File'

  # Step 2: Publish only if the report exists
  - task: PublishBuildArtifacts@1
    condition: and(succeeded(), eq(variables['SONAR_REPORT_EXISTS'], 'true'))
    inputs:
      pathToPublish: '$(Build.ArtifactStagingDirectory)/sonar-report.html'
      artifactName: 'SonarCloudHTMLReport'
      publishLocation: 'Container'
    displayName: 'Publish SonarCloud HTML Report'

Please help us to get the scanning report of the analysis.

Thanks in advance.

I would like to run sonarqube scan on Synapse notebooks in Azure DevOps. In our Synapse notebooks, we have python, Pyspark, SQL and R codes. Our objective is to scan for vulnerabilities, code smells, bugs, etc.

We have write a code in DevOps for the same. Somehow, it is not addressing our objectives.

trigger: none
# - main
# - master

pool:
  vmImage: ubuntu-latest

steps:
  - checkout: self

  - script: |
      echo "Setting the working directory"
      cd $(Build.SourcesDirectory)/notebook
      echo "Current working directory: $(pwd)"
      ls -R
    displayName: 'Change Working Directory to notebook'

  - task: SonarCloudPrepare@3
    inputs:
      SonarQube: 'SonarCloud'
      anization: '****'
      scannerMode: 'cli'
      configMode: 'manual'
      cliProjectKey: 'ABCD'
      cliProjectName: 'ABCD'
      cliSources: 'notebook'  # Ensure this path is correct
      extraProperties: |
        sonar.sources=notebook
        sonar.inclusions=notebook/*.json
        sonar.verbose=true
        sonar.python.version=3.x  # Specify Python version
        sonar.language=py  # Only one language is allowed
        sonar.report.export.path=$(Build.ArtifactStagingDirectory)/sonar-report.html

  - script: |
      echo "Verifying notebook directory contents"
      ls -R $(Build.SourcesDirectory)/notebook
    displayName: 'Verify Working Directory'

  - task: SonarCloudAnalyze@3
    inputs:
      jdkversion: 'JAVA_HOME_17_X64'

  - script: |
      echo "Fetching SonarCloud analysis report..."
      curl -u $SONAR_TOKEN: -X GET "https://sonarcloud.io/api/issues/search?componentKeys=BPEODE_dssynapseprd" -o $(Build.ArtifactStagingDirectory)/sonar-results.json
      jq '.' $(Build.ArtifactStagingDirectory)/sonar-results.json > $(Build.ArtifactStagingDirectory)/sonar-report.html
    displayName: 'Export SonarCloud Results'

  - script: |
      echo "Listing files in ArtifactStagingDirectory..."
      ls -l $(Build.ArtifactStagingDirectory)/
    displayName: 'Debug - Verify Sonar Report File'

  # Step 1: Verify SonarCloud Report Exists
  - script: |
      if [ -f "$(Build.ArtifactStagingDirectory)/sonar-report.html" ]; then
        echo "##vso[task.setvariable variable=SONAR_REPORT_EXISTS]true"
      else
        echo "Sonar report not found!"
        echo "##vso[task.setvariable variable=SONAR_REPORT_EXISTS]false"
      fi
    displayName: 'Check Sonar Report File'

  # Step 2: Publish only if the report exists
  - task: PublishBuildArtifacts@1
    condition: and(succeeded(), eq(variables['SONAR_REPORT_EXISTS'], 'true'))
    inputs:
      pathToPublish: '$(Build.ArtifactStagingDirectory)/sonar-report.html'
      artifactName: 'SonarCloudHTMLReport'
      publishLocation: 'Container'
    displayName: 'Publish SonarCloud HTML Report'

Please help us to get the scanning report of the analysis.

Thanks in advance.

• azure-devops
• azure-pipelines
• sonarqube
• sonarqube-scan
• azure-devops-extensions

• 1 So what is happening? Any relevant errors or outputs? – shotor Commented Feb 3 at 3:53
• 1 As per the link, it could not directly support html report. May i know if you confirm the result exist on sonarqube side? And please clarify which step in devops you are failed to get the report. – wade zhou - MSFT Commented Feb 3 at 4:05
• Thank you very much for your response. Pipeline is getting succeeded and artifacts are also available. However, it is empty file. – RSK Commented Feb 3 at 4:26
• Check the sonarqube extension doc for devops, it typically use Publish Quality Gate Result task(SonarCloudPublish@3) to publish the result to build summary page. What if you use this task? And have you confirm the result on sonarqube side? – wade zhou - MSFT Commented Feb 3 at 9:19
• @RSK, i followed your task to retrieve the sonarcloud results. It appears you didn't define $SONAR_TOKEN but directly invoked it. Try to fix with variable syntax and it works for me. Please check the details below, thanks. – wade zhou - MSFT Commented Feb 14 at 8:48

1 Answer 1

I can reproduce the same with your task. This is because you didn't define $SONAR_TOKEN properly in your yaml.

In Export SonarCloud Results, please fix the format of $SONAR_TOKEN as $(SONAR_TOKEN), so that the token can be invoked correctly.

The sonar-report.html in artifact contains the content correctly: