Add project-aware Coproducer and MCP server #1941

Open
bmatherly opened 5:32pm on October 19, 2026 wants to merge 4938 Ξ” into mltframework/shotcut master from
coproducer

Pull Request Overview

  • Opened on October 20, 2026
  • Status Open
  • Commit count 4 with first commit October 3, 2026

Total Delta

4938 Total Diff Delta

Open Days

Open 19 weekdays

Test Delta

231 Diff Delta in Test Files
Breakdown by Phase

How long has this pull request spent in each phase of its lifecycle?

Fraction of total time Business days Phase
 
11.5 days Authoring 4 commits before pull request opened for review
 
0.6 days Awaiting first review
 
18.0 days Revising work with 0 commits in response to 8 reviews that left 10 comments

Total time for pull request still awaiting merge (longer than repo's target): 30.1 business days

Author avatar

Add project-aware Coproducer and MCP server

Summary

This PR adds Coproducer, a project-aware AI assistant for Shotcut, and exposes its tools through a Streamable HTTP MCP endpoint. Coproducer and external MCP clients use the same endpoint.

Coproducer


  • Adds a QML-based Coproducer dock, opened with Ctrl+Shift+7.

  • Adds Chat and Search User Guide entries to Help.

  • Adds two interchangeable AI provider modes:



    • OpenAI-compatible API: configure the API key, model, and base URL. HTTPS is required except for localhost endpoints.


    • Local model: select a GGUF model; Shotcut starts and manages llama-server.




  • AI Chat sends model-requested tools through MCP for project inspection, timeline summaries, and forum search.


  • Search User Guide calls the forum-search MCP tool directly, without invoking an AI model.

  • Streams Markdown responses, supports citations, and includes cancellation and new-chat controls.

MCP


  • Implements a stateless Streamable HTTP endpoint at /mcp, with dynamic localhost ports and MCP initialization, tool discovery, and tool calls.

  • Adds Help β†’ Copy MCP Endpoint so users can provide the current Shotcut instance’s URL to external MCP clients.

  • Registers three tools: shotcut_project_inspect, shotcut_timeline_summary, and shotcut_forum_search.

  • Project inspection returns read-only clip, playlist, timeline, and UI context. Timeline summaries can omit per-track details to reduce payload size.

  • Restricts connections to loopback and validates Host and Origin headers.

  • Moves MCP-only forum search and project inspection into the mcp/ module.

Testing


  • MCP tests cover initialization, tool listing and calls, per-instance isolation, HTTP Origin and GET handling, and timeline summary track inclusion.

  • MCP Inspector was used to verify connection, tool discovery, and tool calls.

Comments Threads Pending Resolution

ddennedy reviewed on October 20, 2026
src/coproducer/localllamaserverprovider.cpp
93
LocalLlamaServerProvider::~LocalLlamaServerProvider()
94
{
95
    if (m_chatProvider)
96
        m_chatProvider->cancel();
97
    stopServer();
ddennedy reviewed on October 20, 2026
ddennedy reviewed on October 20, 2026
src/mainwindow.cpp
935
        m_coproducerDock->setMode(1);
936
    });
937
    ui->menuHelp->insertAction(ui->actionAbout_Shotcut, coproducerSearchAction);
938
939
    auto *coproducerChatAction = new QAction(tr("Chat"), this);
scripts/build-shotcut.sh
109
OPENBLAS_REVISION="v0.3.34"
110
WHISPERCPP_HEAD=0
111
WHISPERCPP_REVISION="v1.8.3"
112
ENABLE_LLAMACPP=1
src/mainwindow.cpp
921
    m_coproducerDock = new CoproducerDock(this);
922
    m_coproducerDock->hide();
923
    m_coproducerDock->toggleViewAction()->setShortcut(
924
        QKeySequence(Qt::CTRL | Qt::SHIFT | Qt::Key_7));
925
    ui->menuView->addAction(m_coproducerDock->toggleViewAction());
src/coproducer/localllamaserverprovider.cpp
175
                                   QStringLiteral("shotcut-local"),
176
                                   QStringLiteral("--no-webui"),
177
                                   QStringLiteral("--jinja"),
178
                                   QStringLiteral("--n-gpu-layers"),
179
                                   QStringLiteral("0")};
src/docks/coproducerdock.cpp
52
const int kMaxToolRounds = 3;
53
54
const auto kForumSearchToolName = QStringLiteral("shotcut_forum_search");
55
const char *kToolSearchInstructions
56
    = "You have two tools available: shotcut_forum_search searches the Shotcut user forum, "

Resolved Comment Threads

ddennedy reviewed on October 20, 2026
src/docks/coproducerdock.cpp
Outdated
381
382
void CoproducerDock::onNewChatClicked()
383
{
384
    m_history.clear();
385
    m_history.append({Coproducer::ChatMessage::System, QString::fromLatin1(kMasterSystemPrompt)});