Feature request: ".sam" folder to encapsulate SAM CLI's project metadata

[sanathkr]

Capturing the discussion from a separate thread as a feature request for a .sam folder in project level to encapsulate SAM CLI's metadata such as build artifacts, packaged template etc. Similar to .samrc, we need to design this to make sure we keep the semantics and usage of this folder clear.

[Need Help/Owner 👋]: As next step, create a design document following the design document template (/designs/_template.md) and send a pull request for discussion before implementing.

FAQs

Can it exist in multiple places (project root & user home)?

No, just project root. .sam folder will contain build artifacts and it's project specific; moving to user home brings unwanted side effects (disk space and logic complexity).

Q: Can other tools consume the contents of this folder?

.sam folder contains read-only build artifacts so any tool built to work with sam can read from that.

Q: How does sam cli operate in absence of this folder?

We should create it in the absence of it - That provides a Just Works(tm) experience. Here's a sample scenario for this:

No explicit sam build from the user

UX

This is the ideal scenario assuming .sam folder contains artifacts, packaged-template and other commands are aware of .sam and .samrc:

• Initialize boilerplate: sam init --name my-project && cd my-project
• Deploy: sam deploy

SAM application will be deployed seamless and the entire process logged. One could introduce sam local before sam deploy and sam build will run seamlessly if .sam folder is absent.
Power users can then modify the convention and opinions if they wanted to.

Logic

• sam deploy checks .sam absence and run sam build for the runtime chosen in template.yaml
• sam deploy then run sam package
• sam package creates a S3 bucket based on project name (.samrc) if one doesn't exist
• sam deploy ultimately deploys the application
• .samrc is instrumental in allowing the user to add his/her opinions to change this flow (e.g. custom build, custom stack name, custom s3 bucket name)

Pros

1. Seamless experience and just works without having to know built-template.yaml, packaged-template.yaml, etc.
2. New users don't have to learn upfront how Lambda packaging works and embed sam local within their package managers
   • npm run build -> sam build
   • npm run deploy -> sam deploy
   • make build -> sam build
   • make deploy -> sam deploy
3. Power users can rely on .sam and build wrappers/tools until sam package/deploy changes its logic

Cons

1. This can only happen post-MVP as sam local, sam package, and sam deploy would need to check for .sam

Q: How do CI/CD systems use this folder to extra artifacts etc.

Extra artifacts is a responsibility of the CI system and SAM only takes into account the application artifact so CloudFormation can deploy it afterwards - But perhaps I didn't understand the question.

[Future] Here's an example using CodeBuild where CodePipeline would pass that as an Input Artifact to CloudFormation:

phases:
  install:
    commands:
      - echo "[Security phase]"
      - pipenv check # run some CVE checks on deps
  pre_build:
    commands:
      - echo "[Pre-Build phase] Nothing to be done as SAM build now handles that for me ;)"
  build:
    commands:
      - echo "[Build phase] Starting SAM build `date` in `pwd`"
      - sam build
      - echo "[Build phase] SAM build completed on `date`"
  post_build:
    commands:
      - echo "[Post-build] Uploading artifacts to S3 `date`"
      - sam package

artifacts:
  files:
    - .sam/template-packaged.yaml
  discard-paths: yes

[Post-MVP] Here's the same example but without changing sam package/deploy yet:

phases:
  install:
    commands:
      - echo "[Security phase]"
      - pipenv check # run some CVE checks on deps
      - aws s3api head-bucket --bucket $BUILD_OUTPUT_BUCKET || aws s3 mb s3://$BUILD_OUTPUT_BUCKET
  pre_build:
    commands:
      - echo "[Pre-Build phase] Nothing to be done as SAM build now handles that for me ;)"
  build:
    commands:
      - echo "[Build phase] Starting SAM build `date` in `pwd`"
      - sam build
      - echo "[Build phase] SAM build completed on `date`"
  post_build:
    commands:
      - echo "[Post-build] Uploading artifacts to S3 `date`"
      - echo "Runs sed/awk/whatever needed to replace CodeUri in template.yaml to .sam/functionLogicalName/"
      - sam package --template-file template.yaml --s3-bucket $BUILD_OUTPUT_BUCKET --output-template-file .sam/template-packaged.yaml

artifacts:
  files:
    - .sam/template-packaged.yaml
  discard-paths: yes

By supporting .sam folder and only storing artifacts this can set the path for a fast follow up on sam package and sam deploy afterwards.

[mvanbaak]

isn't this .aws-sam now?

[sanathkr]

Yes it is :) I am going to keep the issue open because it is implemented only for sam build and sam package. We have a long ways to go to fully adopt this experience.

[awsjeffg]

Closing this as finished.