The Data Science Cycle on SeaGit: a Beginner’s Guide

From raw documents to an AI app you can change without fear: which stack does each step, how the stacks connect, and the exact order to follow. No prior MLOps experience needed.

Select a step to jump to it. Swipe the diagram sideways to see all of it.

The cycle in one table

Step	What you do	Stack
1. Data	Put documents in a bucket, or keep structured data in a database.	RAG stack uploads bucket, Database, Redis
2. Explore	Try ideas in Python and call models through one gateway.	Notebooks + LLM Gateway
3. Track	Record the settings, scores and models of each try.	MLflow
4. Build	Turn the idea into a running service.	RAG stack, or your own app from GitHub
5. Evaluate	Score the app on a set of questions with known answers.	Evals stack (results in MLflow)
6. Release	Ship only what passes; a failing change is stopped.	Promote to main, or Adopt a settings change
7. Monitor	See real questions, answers, cost and user feedback.	MLflow traces, LLM Gateway usage
8. Improve	Fix what you saw, and go back to Evaluate.	Change settings, or push new code

Steps 5 to 8 repeat for as long as the app lives: every change is scored before it reaches your users.

How the stacks connect

You never type an address or a password to connect two stacks: each form lists the stacks you already have, and SeaGit wires them together.

From	To	Why
Auth Proxy	MLflow, Notebooks	One sign-in with your own identity provider in front of the browser tools.
Notebooks	MLflow	Turn on Connect to MLflow: every notebook can log runs, metrics and models with no setup.
Notebooks, RAG stack, your app	LLM Gateway	One OpenAI-compatible address for every approved model, with a key and budget per team.
RAG stack, LLM Gateway	MLflow	Turn on tracing: each question, answer, token count and feedback becomes an MLflow trace.
Evals stack	RAG stack or your app	Asks your test questions and scores the answers.
Evals stack	MLflow	Records every evaluation run: each question, answer and score.
SeaGit	Evals stack	Asks for a verdict before a deployment is promoted or a settings change is adopted, and blocks or warns on a fail.

For the full architecture diagram, see the MLOps Platform page.

Before you start

A paid plan.
An EKS cluster in an environment with a domain (AWS only).
Amazon Bedrock access for the models you want, approved for your organization in SeaGit.
Create every stack in the same environment and cluster; each form only lists the stacks it can reach.
Every stack starts from Applications → Templates.

Follow these steps

Step 1. Create an Auth Proxy stack. It is the sign-in page for your browser tools; one is enough for the whole team.

Register an OAuth app in your identity provider (for example GitHub or Google) and enter its client ID and secret.
Choose who may sign in: an organization, team, group or email domain.

Create MLflow

Step 2. Create an MLflow stack and choose the Auth Proxy for sign-in. This is where every result of every step lands.

Keep Create new Postgres: MLflow stores experiments, runs and the model registry there.
Open MLflow from the stack’s Stack tab. It is empty for now.

Create the LLM Gateway

Step 3. Create an LLM Gateway stack and pick the models your organization approved. Turn on tracing and pick the MLflow stack.

Every app and notebook calls models through this one address with its own key.
Budgets and usage are per key, so you see what each team spends.

Explore in Notebooks

Step 4. Create a Notebooks stack and turn on Connect to MLflow. Each person gets their own Jupyter server.

Load your data, try prompts and models through the gateway, and compare ideas.
Log each try with mlflow.log_param and mlflow.log_metric, so the best one is easy to find later.

Build a RAG app

Step 5. Create a RAG stack: pick uploads, the chat model, and optionally the LLM Gateway. Turn on tracing and pick the MLflow stack.

Upload a few .md or .pdf files to the stack’s bucket. They are indexed within seconds.
Open the web UI and ask a question: answers cite the documents they came from.
Building your own app instead? Connect its GitHub repository as an application; each push deploys a non-main copy, so main is never overwritten.

Write golden questions

Step 6. A golden set is a list of questions with the answer you expect, one JSON object per line. It is the test your app must pass.

Start with 10 to 30 questions your users really ask.
Keep the expected answer short: the contains score checks that it appears in the answer.

For example, save this as golden.jsonl:

{"input": "Which vector database does the RAG stack use?", "expected": "Qdrant"}
{"input": "Which model embeds the documents?", "expected": "Titan"}
{"input": "How long can I roll back an adopted change?", "expected": "23 hours"}

Create the Evals stack

Step 7. Create an Evals stack: pick the RAG stack (or your app) to score and the MLflow stack to record in. Choose the gate: Block stops a failing release, Warn asks you to confirm.

Run an evaluation: the verdict and scores appear in SeaGit, and the full run in MLflow.
Set the thresholds, for example contains at least 0.85, once you have seen a few runs.

Change and release safely

Step 8. On the RAG stack’s Stack tab, choose Change settings and try new values such as top-k or chunk size. A candidate runs next to the live stack and is scored; your users see nothing yet.

Change settings is switched on per account: if you do not see it, ask SeaGit to enable it for yours.
Passed: choose Adopt, and live traffic moves to the new settings.
Failed: under Block you cannot adopt; choose Discard to remove the candidate and its index.
Changed your mind after adopting? Roll back within 23 hours; the previous index is kept for 24.
For your own app, the same gate runs when you promote a deployment to main.

Monitor and improve

Step 9. Open MLflow to read real questions and answers, and the LLM Gateway for cost per key. When something looks wrong, change a setting or push a fix, and the cycle starts again at Evaluate.

Thumbs up and down from the RAG web UI are stored with each trace.
Delete stacks you only used for testing: they use cluster capacity.

FAQ

Do I need every stack?

No. The smallest useful loop is a RAG stack, an MLflow stack and an Evals stack. Add the LLM Gateway when several apps or teams share models, Notebooks when people experiment in Python, and the Auth Proxy when people open MLflow or Notebooks in a browser.

In what order should I create the stacks?

Auth Proxy, MLflow, LLM Gateway, Notebooks, then your app, then Evals. Each form lists the stacks created before it, so you connect them by picking one from a list.

What does a beginner need to write?

Only the golden questions: one line per question with the answer you expect. Every stack is a form with sensible defaults.

What happens when a change scores worse?

With a Block gate it cannot be adopted or promoted; discard it, fix it and try again. With a Warn gate you are asked to confirm.

Deploy a RAG stack in minutes — step 5 in detail, with screenshots.
How to evaluate a RAG or LLM app — golden questions, scorers and the gate.
Change RAG settings safely — every setting you can change and how adopt and roll back work.
Demo: an evaluated RAG release — a real run where the gate caught a regression.