Skip to content

A simple, persistent, process-based job queue server in Go. Runs jobs as OS processes, with REST API, webhooks, and Docker support.

Notifications You must be signed in to change notification settings

The-Focus-AI/shell-job-queue

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🛠 Process-Based Job Queue in Go

This is a simple, file-backed job queue server written in Go. Each job runs as a separate OS-level process. Useful for wrapping command-line tools like model runners, converters, or custom scripts.

✅ Features

  • Jobs are executed as child processes
  • Each job logs stderr and stores stdout as the final result
  • Persistent metadata and logs saved to the filesystem
  • Webhook support to notify external services on job completion
  • REST API for job submission, status tracking, result fetching, and cancellation

🚀 Usage

1. Build and Run Locally

make build
./processjobqueue

Or run directly:

make run

2. Submit a Job

curl -X POST http://localhost:8080/jobs \
  -H 'Content-Type: application/json' \
  -d '{
    "args": ["echo", "Hello, world!"],
    "mime_type": "text/plain",
    "webhook": "https://webhook.site/your-id"
  }'

3. Check Status

curl http://localhost:8080/jobs/<job-id>/status

4. Get Result

curl http://localhost:8080/jobs/<job-id>/result

5. Cancel a Job

curl -X PUT http://localhost:8080/jobs/<job-id>/cancel

🐳 Docker

Build

make docker-build

Run

make docker-run

Mounts the jobs/ folder for persistent storage and serves the API on port 8080.


📁 Job Directory Structure

Each job is stored in:

jobs/<job-id>/
├── meta.json      ← job status + metadata
├── stdout.txt     ← final output
└── stderr.txt     ← logs (live updates)

🧩 Requirements

  • Go 1.21+
  • (Optional) Docker for containerized deployment

📬 TODOs

  • Retry + sign webhooks
  • Job priorities or delayed execution
  • Streaming logs over SSE

About

A simple, persistent, process-based job queue server in Go. Runs jobs as OS processes, with REST API, webhooks, and Docker support.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published