Perform snapshot testing using `md` files

[tusharmath]

Currently, our test suite is all over the place with various ways of writing and asserting test. I would like to propose an easier way of writing, executing and updating tests.

Proposal

• Use Insta Snapshots to create and manage snapshot of outputs.

• Use a markdown file to write tests using a mix of GraphQL, YAML or JSON syntax.

  ---

  config:

  type Query {
    greetings: String @const(data: "Hello World!")
  }

  assert:

  assert:
    - request:
        method: POST
        url: http://localhost:8080/graphql
        body:
          query: "query { greetings }"
      response:
        body:
          data:
            greetings: Hello World

  ---

Technical Requirements

• Write a module that can convert all existing tests automatically to this new format. It can read the existing YAML files and generate a combined .md with the config embedded it it.
• In the first run a series of snapshots should be generated:
  • Client SDL snapshots
  • Expected response snapshots
• This approach should allow us to merge http_spec and graphql_spec into one spec. Call it execution_spec.
• The md file may or may not have an assertion block in it. If it does have an assert block, use it to run the execution spec.
• We could have more than one assertion in an MD file.
• Do not touch the existing specs. We will delete them after a month of regression.
• Make sure there is proper logging of what tests passed and what has failed.
• There is an ability to run only one test using an annotation, make sure that also works in markdown.

Links
Youtube: https://www.youtube.com/watch?v=rCHrMqE4JOY&t=2s

[tusharmath]

/bounty 350$

[algora-pbc]

💎 $350 bounty created by tailcallhq
🙋 If you'd like to work on this issue, comment below to get assigned
👉 To claim this bounty, submit a pull request that includes the text /claim #983 somewhere in its body

📝 Before proceeding, please make sure you can receive payouts in your country.
💵 Payment arrives in your account 2-5 days after the bounty is rewarded.
💯 You keep 100% of the bounty award.
👍 If you go above and beyond by cleaning up or enhancing aspects beyond the scope of the PR, a tip may be awarded as a token of appreciation.
🙏 Thank you for contributing to tailcallhq/tailcall!.
🙋‍♂️ Join our discord channel if you need help.

👉 Add a bounty • Share on socials

[wesleymatosdev]

/attempt #983

Algora profile	Completed bounties	Tech	Active attempts	Options
@ologbonowiwi	16 tailcallhq bounties + 19 bounties from 5 projects	TypeScript, Rust, Shell & more		Cancel attempt

[algora-pbc]

@ologbonowiwi: The Tailcall Inc. team prefers to assign a single contributor to the issue rather than let anyone attempt it right away. We recommend waiting for a confirmation from a member before getting started.

[tusharmath]

@ologbonowiwi Are you working on this or should we release it for other users to attempt?

[wesleymatosdev]

I took the weekend, so didn't have much time to look at it since it started. I had some ideas for it, but I'm still working on the @call issue, so we can likely open this and if no one takes up, I can work on it again after finishing #846.

[mogery]

I'd love to work on this.

[algora-pbc]

💡 @mogery submitted a pull request that claims the bounty. You can visit your bounty board to reward.

[algora-pbc]

🎉🎈 @mogery has been awarded $350! 🎈🎊

• Share on socials
• Give feedback