I have a GitLab CI/CD Pipeline, which has a job that performs terraform plan, which creates plan file as an artifact to be stored for 20 minutes. This job gets launched when Merge Request to "master" branch is created. And after the the job passes the merge button becomes available to merge into the master. There is also additional job that runs the terraform apply plan_file intended for terraform resource creation that runs after the merge request.

However, for some reason this job can not find the needed file. Why can this be ? I thought according to GitLab that artifacts are available to all the jobs. It throws such an error:

The CI/CD Pipeline YAML file looks like this:

stages:
  - analysis
  - plan
  - deployment
  - release

terraform_validate_configuration:
  stage: analysis
  image:
    name: "hashicorp/terraform:1.10"
    entrypoint: [""]
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  script:
    - terraform init
    - terraform validate
  artifacts:
    paths:
      - ./.terraform/
    expire_in: "3 mins"

checkov_scan_directory:
  stage: analysis
  image:
    name: "bridgecrew/checkov:3.2.344"
    entrypoint: [""]
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  script:
    - checkov --directory ./ --soft-fail

trivy_scan_security:
  stage: analysis
  image: 
    name: "aquasec/trivy:0.58.2"
    entrypoint: [""]
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  script:
    - trivy config --format table ./

terraform_plan_configuration:
  stage: plan
  image:
    name: "hashicorp/terraform:1.10"
    entrypoint: [""]
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  dependencies:
    - terraform_validate_configuration
  script:
    - terraform init
    - terraform plan -out=deployment_plan
  artifacts:
    paths:
      - ./deployment_plan
    when: on_success
    expire_in: "20 mins"

deploy_terraform_infrastructure:
  stage: deployment
  image:
    name: "hashicorp/terraform:1.10"
    entrypoint: [""]
  rules:
    - if: $CI_COMMIT_BRANCH == "master"
  dependencies:
    - terraform_plan_configuration
    - terraform_validate_configuration
  script:
    - terraform apply deployment_plan

I have a GitLab CI/CD Pipeline, which has a job that performs terraform plan, which creates plan file as an artifact to be stored for 20 minutes. This job gets launched when Merge Request to "master" branch is created. And after the the job passes the merge button becomes available to merge into the master. There is also additional job that runs the terraform apply plan_file intended for terraform resource creation that runs after the merge request.

However, for some reason this job can not find the needed file. Why can this be ? I thought according to GitLab that artifacts are available to all the jobs. It throws such an error:

The CI/CD Pipeline YAML file looks like this:

stages:
  - analysis
  - plan
  - deployment
  - release

terraform_validate_configuration:
  stage: analysis
  image:
    name: "hashicorp/terraform:1.10"
    entrypoint: [""]
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  script:
    - terraform init
    - terraform validate
  artifacts:
    paths:
      - ./.terraform/
    expire_in: "3 mins"

checkov_scan_directory:
  stage: analysis
  image:
    name: "bridgecrew/checkov:3.2.344"
    entrypoint: [""]
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  script:
    - checkov --directory ./ --soft-fail

trivy_scan_security:
  stage: analysis
  image: 
    name: "aquasec/trivy:0.58.2"
    entrypoint: [""]
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  script:
    - trivy config --format table ./

terraform_plan_configuration:
  stage: plan
  image:
    name: "hashicorp/terraform:1.10"
    entrypoint: [""]
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  dependencies:
    - terraform_validate_configuration
  script:
    - terraform init
    - terraform plan -out=deployment_plan
  artifacts:
    paths:
      - ./deployment_plan
    when: on_success
    expire_in: "20 mins"

deploy_terraform_infrastructure:
  stage: deployment
  image:
    name: "hashicorp/terraform:1.10"
    entrypoint: [""]
  rules:
    - if: $CI_COMMIT_BRANCH == "master"
  dependencies:
    - terraform_plan_configuration
    - terraform_validate_configuration
  script:
    - terraform apply deployment_plan

• gitlab
• continuous-integration
• gitlab-ci

• 2 dont these jobs run in separate pipelines? your plan job running gin your merge pipeline and your deploy job running in your push pipeline? – Chris Doyle Commented Jan 17 at 17:38
• True, they are indeed separate pipelines. – Mykoliux Commented Jan 17 at 17:44

1 Answer 1

Turns out these jobs run in separate pipelines and the problem of transferring artifacts between different pipelines has been encountered before (Gitlab CI/CD Pass artifacts/variables between pipelines).