Files
joplin-cli-gateway/language-framework-analysis.md
2026-08-19 17:18:45 +01:00

26 KiB
Raw Permalink Blame History

Programming Language and Framework Analysis

Status: accepted implementation decision, 2026-08-19. TypeScript, Node.js, Fastify, and Mercurius were selected, subject to continued compatibility testing of the experimental Joplin CLI Data API boundary.

Executive Conclusion

The recommended starting stack is:

  • TypeScript on a current Node.js LTS release;
  • Fastify for REST, request validation, lifecycle hooks, and HTTP concerns;
  • Mercurius for GraphQL, with GraphQL Yoga as the leading alternative;
  • an explicit single-consumer profile actor/operation queue shared by REST, GraphQL, background sync, and future event delivery;
  • a replaceable JoplinAdapter boundary that initially treats Joplin CLI as an external program or local service;
  • JSON repositories for MVP administrator configuration and generated state, behind interfaces designed for later SQLite repositories.

Go is the strongest alternative. If an integration spike proves that the gateway can rely entirely on documented CLI commands or the local Joplin Data API, with no need to load Joplin libraries, Go becomes nearly as attractive and may be preferable for process supervision and a small operational binary.

The most important technical work before final commitment is therefore not an HTTP prototype. It is a narrow Joplin adapter spike that proves how mandatory pre-operation sync, local data operations, post-operation sync, trash, revisions, and exclusive profile ownership can coexist.

Constraints That Actually Drive the Choice

This is not primarily a high-throughput web service. Its defining constraints are:

  1. One gateway deployment owns one Joplin CLI profile.
  2. No two processes may access that profile concurrently.
  3. Every content request enters one serial queue.
  4. Reads require a successful pre-operation sync.
  5. Mutations require a post-operation sync and can produce degraded local success.
  6. REST is normative; GraphQL exposes the same application services.
  7. Future WebSocket/subscription delivery must not bypass permissions or the durable event cursor.
  8. Joplin operations and network sync will dominate latency; raw framework throughput is secondary.
  9. The container already needs the Joplin CLI and therefore a compatible Node runtime, regardless of the gateway language.
  10. The public API must remain stable if the internal Joplin adapter changes.

The official terminal documentation describes Joplin CLI's server command as experimental and warns that it must use a separate profile so concurrent CLI instances do not access the same profile. It also exposes joplin sync as a separate command. See Joplin Terminal Application.

The Critical Integration Unknown

There are four plausible ways to control Joplin. Language selection depends partly on which one proves viable.

1. One-shot CLI commands

The gateway serially executes joplin --profile ... sync and individual CLI commands.

Advantages:

  • strongest process and licensing boundary;
  • uses documented CLI behaviour;
  • easy to enforce one-operation-at-a-time ownership;
  • works from any gateway language.

Disadvantages:

  • process startup on every step;
  • CLI output is not uniformly designed as a stable machine protocol;
  • not every required operation has convenient structured output;
  • command parsing and error classification may be brittle;
  • a request can require sync, one or more data commands, and another sync.

This approach is acceptable for an MVP only if the adapter spike demonstrates complete, deterministic coverage.

2. Joplin CLI's experimental local Data API server

The gateway starts joplin server start on a dedicated profile and calls its local REST Data API.

Advantages:

  • JSON operations already exist for notes, folders, tags, search, revisions, and events;
  • avoids parsing most human-oriented CLI output;
  • supports richer queries than many shell commands;
  • language-neutral HTTP boundary.

Disadvantages:

  • Joplin explicitly labels this CLI server experimental;
  • the Data API does not expose a documented “sync now and wait” endpoint;
  • running a separate joplin sync process against the same profile would violate the exclusive-profile rule;
  • periodic background sync alone does not satisfy this gateway's sync-before-every-operation contract;
  • trash restoration and reconstructed revision restoration may still require functionality outside the raw Data API.

This is the most important spike. The project must establish whether the same long-running Joplin process can safely perform an explicit sync on demand, or whether it would have to be stopped and restarted around sync operations.

3. Embed Joplin packages in the gateway process

A TypeScript gateway imports Joplin application/library packages and invokes models and synchronisation services directly.

Advantages:

  • potentially the cleanest access to sync, revisions, trash, models, and events;
  • one process can own the profile and queue;
  • avoids text parsing and local HTTP indirection;
  • shares Joplin's native types and behaviour.

Disadvantages:

  • these are application internals, not a documented stable gateway SDK;
  • upgrades can break the gateway even across ordinary Joplin releases;
  • Joplin's app CLI depends on a large, native-module-heavy package graph;
  • initialization assumes Joplin application lifecycle and global registries;
  • the CLI/package is AGPL-3.0-or-later, so embedding or tightly linking packages requires licensing review and appropriate compliance;
  • this option effectively forces Node/TypeScript.

Joplin's current CLI source imports @joplin/lib and related internal packages, and its package metadata declares AGPL-3.0-or-later. See the official packages/app-cli/package.json.

This should be a last resort or a deliberate maintained integration, not an accidental shortcut.

4. A dedicated Joplin helper process

A small Node process owns the Joplin profile and presents a private structured protocol to the gateway. It may wrap Joplin internals or a maintained Joplin patch while the public gateway remains language-neutral.

Advantages:

  • isolates Joplin coupling and crashes;
  • gives the main gateway a stable private adapter protocol;
  • permits Go, Rust, Python, or another language for the public service;
  • can serialize all profile operations in exactly one owner.

Disadvantages:

  • two components and a private protocol to maintain;
  • still inherits Joplin internal compatibility and licensing concerns if it imports Joplin packages;
  • more complex lifecycle, health, logging, and packaging.

This is a credible post-spike fallback if neither one-shot commands nor the experimental server can satisfy the contract cleanly.

Evaluation Criteria

The following matrix uses a 15 score. It is a project-specific decision aid, not a general language ranking.

Criterion Weight TypeScript Go Rust C# Kotlin/Java Python Elixir
Joplin/runtime alignment 25% 5.0 4.0 4.0 3.5 3.5 4.0 3.5
Process ownership and concurrency 20% 4.0 5.0 5.0 4.5 4.0 4.0 4.5
REST, GraphQL, realtime fit 20% 5.0 4.0 4.0 5.0 5.0 4.5 4.0
Correctness and maintainability 15% 4.0 5.0 5.0 5.0 5.0 4.0 4.0
Container/deployment simplicity 10% 4.5 4.0 4.5 3.0 3.0 3.5 3.0
Relevant libraries and testing 10% 5.0 4.0 3.5 5.0 5.0 5.0 3.5
Weighted result 4.60 4.35 4.35 4.33 4.23 4.15 3.83

The small gaps are intentional. Team experience can legitimately move the top four choices. The adapter result matters more than a few tenths in this matrix.

Language Analysis

Strengths

  • Joplin CLI is itself a Node/TypeScript application, so the runtime already exists in the container.
  • It preserves every adapter option, including a future helper or direct library experiment.
  • Async subprocess control is mature. Node's child_process.spawn() runs asynchronously without blocking the event loop, while direct argument arrays avoid a shell. See Node child processes.
  • A promise-based single-consumer queue maps naturally to exclusive profile ownership.
  • The ecosystem is particularly strong for JSON APIs, JSON Schema, OpenAPI, GraphQL, WebSocket protocols, structured logging, and test doubles.
  • Sharing DTOs, permission types, error codes, event envelopes, and domain services between REST and GraphQL is straightforward.
  • Future SDK generation is natural from OpenAPI and GraphQL schemas.

Weaknesses

  • The type system disappears at runtime, so every external boundary still requires validation.
  • Package churn and transitive dependency volume need active control.
  • Synchronous CPU or database work can block the event loop. This service is mostly I/O-bound, but event filtering and future SQLite work must still be bounded.
  • It is easy to accidentally couple HTTP handlers directly to Joplin commands without a disciplined application layer.
  • Runtime alignment may tempt the project to import undocumented Joplin internals prematurely.

Suitability

Very high. The workload is I/O-heavy, serialized at the Joplin boundary, and rich in JSON/protocol concerns. Node's single event loop is not a disadvantage when the design intentionally funnels Joplin work through one actor.

Go — Strongest Alternative

Strengths

  • Goroutines and channels are an excellent fit for a profile actor: one goroutine owns the Joplin queue and request handlers communicate with it through typed messages.
  • os/exec executes programs directly without implicitly invoking a shell, which is a strong default for safely passing Joplin IDs and content paths. See os/exec.
  • Cancellation and timeouts flow cleanly through context.Context.
  • Strong compile-time types, simple deployment, fast startup, and predictable memory behaviour.
  • The standard library covers much of HTTP, crypto, process, testing, and SQL functionality.
  • gqlgen produces strongly typed schema-first GraphQL resolvers and supports WebSocket/SSE subscriptions. See gqlgen subscriptions.

Weaknesses

  • The container still needs Node and Joplin, so “single static binary” does not eliminate the second runtime.
  • GraphQL development is more code-generation-heavy than TypeScript, especially while the schema is evolving.
  • JSON unions, flexible GraphQL inputs, and schema iteration are more verbose.
  • Any direct Joplin-library strategy would require a Node helper, splitting the system.
  • OAuth authorization-server libraries and rich API middleware are available, but the selection is less obvious than in the JavaScript, JVM, or .NET ecosystems.

Suitability

Excellent if the Joplin boundary remains subprocess or HTTP. Go would be the preferred choice for a strict two-process architecture with a language-neutral Joplin adapter.

Rust

Recommended stack: Tokio, Axum, async-graphql, SQLx or rusqlite, tracing.

Strengths

  • Strongest compile-time guarantees and explicit ownership.
  • Tokio tasks and channels model the profile actor well.
  • Axum supports modular HTTP routing and the Tower middleware ecosystem; it also provides WebSocket and SSE support. See Axum and Axum WebSockets.
  • Excellent process control, resource limits, and low runtime overhead.
  • SQLx has first-class SQLite support.

Weaknesses

  • Highest implementation cost for an API whose performance is dominated by Joplin sync.
  • GraphQL, OAuth-provider, and high-level API ecosystems are smaller and change more quickly than the TypeScript/.NET/JVM equivalents.
  • More code is required for ordinary product changes and integration glue.
  • No meaningful advantage for direct Joplin integration.

Suitability

Technically excellent, economically hard to justify unless Rust expertise or safety requirements are unusually strong.

C# and ASP.NET Core

Recommended stack: ASP.NET Core Minimal APIs or controllers, Hot Chocolate GraphQL, BackgroundService, System.Threading.Channels, EF Core or a lightweight SQLite layer.

Strengths

  • Excellent HTTP, authentication, dependency injection, background-service, configuration, and testing ecosystem.
  • Channel<T> cleanly represents the single Joplin actor queue.
  • Hot Chocolate has mature GraphQL and subscription support over WebSocket/SSE, with replaceable event providers. See Hot Chocolate subscriptions.
  • Strong type system, cancellation, observability, and long-term maintainability.

Weaknesses

  • Adds the .NET runtime beside Node/Joplin.
  • No special Joplin integration advantage.
  • More framework surface than this personal, single-profile gateway initially needs.
  • Container size and memory are likely higher than Go or a Node-only stack.

Suitability

Very good if .NET is the operator's strongest ecosystem. Otherwise it adds operational weight without solving the key adapter uncertainty.

Kotlin/Java

Recommended stack: Spring Boot with WebFlux or MVC, Spring for GraphQL, coroutines for Kotlin where appropriate, JDBC/JOOQ for future SQLite.

Strengths

  • Deep authentication, authorization, validation, configuration, migration, observability, and test ecosystems.
  • Spring for GraphQL officially supports HTTP, WebSocket, and SSE transports. See Spring for GraphQL.
  • Strong long-term compatibility and excellent structured application architecture.

Weaknesses

  • Considerable startup, memory, framework, and build weight for a single-user gateway.
  • Reactive/WebFlux abstractions add complexity around a fundamentally serialized Joplin actor.
  • Adds a JVM beside Node/Joplin with no direct integration advantage.

Suitability

Robust but disproportionate unless JVM expertise or deployment conventions dominate.

Python

Recommended stack: FastAPI, Pydantic, Strawberry or Ariadne GraphQL, asyncio, SQLAlchemy/SQLModel or sqlite3.

Strengths

  • Fastest path to a readable prototype.
  • FastAPI provides validation, generated OpenAPI, dependency injection, security helpers, and WebSockets. See FastAPI features and FastAPI WebSockets.
  • asyncio.create_subprocess_exec() provides asynchronous shell-free process execution. See Python asyncio subprocesses.
  • Excellent testing and data-processing ecosystem.

Weaknesses

  • Static typing is useful but less strongly enforced across the full application and dependency ecosystem.
  • GraphQL frameworks are less consolidated; choosing one becomes a separate long-term commitment.
  • Multi-worker ASGI deployment is dangerous here because workers would not share the in-memory profile queue. This must be explicitly disabled or externally coordinated.
  • Adds Python beside Node/Joplin.
  • Long-lived connection and task cancellation behaviour requires discipline to avoid orphan work.

Suitability

Good for a prototype, acceptable for production, but not the strongest long-term fit when TypeScript offers similar iteration speed with closer Joplin alignment.

Elixir

Recommended stack: Phoenix, Absinthe GraphQL, Ecto/SQLite or PostgreSQL.

Strengths

  • Outstanding supervision, actor-style process ownership, and persistent-connection support.
  • One GenServer owning the Joplin profile is conceptually ideal.
  • Phoenix channels and fault isolation are strong foundations for future subscriptions.

Weaknesses

  • Smaller ecosystem for this specific CLI/GraphQL/OAuth combination.
  • Adds BEAM plus Node/Joplin.
  • SQLite and external-process integration are less conventional than in the leading choices.
  • GraphQL and REST DTO sharing is less direct.

Suitability

Attractive for a realtime-first system, but realtime is post-MVP and should rest on a durable event journal anyway. It does not outweigh the integration cost here.

  • Plain JavaScript: loses compile-time guarantees precisely where permissions and state transitions need them.
  • Bun or Deno as the primary runtime: Joplin is distributed for Node, so alternative runtimes add compatibility risk without removing Node from the container.
  • C/C++: disproportionate memory-safety and development cost.
  • PHP/Ruby: capable web stacks, but weaker fit for supervised long-running CLI ownership and future subscription connections than the finalists.

TypeScript Framework Analysis

Fastify is small enough that the architecture remains explicit while providing schema-driven validation, serialization, hooks, plugins, and structured logging. Its TypeScript support includes type providers for JSON Schema-based validation. See Fastify TypeScript.

Why it fits:

  • REST is the normative contract and maps naturally to route schemas;
  • JSON Schema can generate OpenAPI and validate both request and response shapes;
  • plugins provide clean composition for authentication, request IDs, error mapping, rate limits, and health;
  • it does not prescribe a database or background-job model;
  • GraphQL can be mounted without moving business logic into resolvers;
  • low framework overhead makes the profile actor visible and testable.

Mercurius is Fastify's GraphQL adapter and supports TypeScript, loaders, batched queries, persisted queries, and subscriptions. See Mercurius.

Why it fits:

  • one Fastify lifecycle and authentication context;
  • GraphQL resolvers remain thin projections over the same application services used by REST;
  • subscriptions provide a future path without adding a second HTTP server;
  • loaders can prevent repeated adapter reads within a GraphQL document;
  • GraphQL batching can perform one pre-operation sync for a complete document.

Risk: Mercurius is closely tied to Fastify. That is acceptable if Fastify is deliberately selected.

GraphQL Yoga — Leading Alternative

Yoga is a strong choice if transport portability and standards-based subscription options are valued over tight Fastify integration. It supports GraphQL subscriptions and SSE, with WebSocket support through graphql-ws. See Yoga subscriptions.

Choose Yoga instead of Mercurius if:

  • GraphQL might later be deployed separately;
  • GraphQL-over-SSE is a likely first realtime transport;
  • framework-neutral request handling matters more than a unified Fastify plugin model.

NestJS offers modules, dependency injection, guards, queues, scheduling, REST, GraphQL, and Fastify integration. Its official GraphQL support works with Apollo or Mercurius. See NestJS GraphQL.

It is a sound choice for a large team or a broad administration platform. For this gateway it initially adds decorators, module lifecycle, dependency-injection conventions, and adapter layers around a domain that is small but operationally unusual. The custom profile actor remains necessary regardless of Nest.

Express and Apollo Server

Both are mature, but neither is the best combined foundation here:

  • Express lacks Fastify's schema-first validation/serialization model.
  • Apollo is GraphQL-focused while REST is normative.
  • Apollo subscriptions require additional transport integration.
  • using separate REST and GraphQL stacks makes shared authentication, error semantics, and lifecycle easier to drift.

These are starting directions, not dependencies to install blindly:

Concern Recommendation Reason
Runtime Current Node LTS Same runtime family as Joplin CLI
Language TypeScript, strict mode Permission and state-machine correctness
Package format ESM unless Joplin adapter proves CommonJS necessary Modern ecosystem; adapter can be isolated
REST Fastify Schema validation, lifecycle, low ceremony
REST schemas TypeBox or authored JSON Schema Runtime validation plus static inference
GraphQL Mercurius, schema-first SDL Explicit public contract and subscriptions path
OpenAPI Generated from Fastify route schemas REST remains normative
CLI execution spawn/execFile with argument arrays, never a shell Injection resistance, cancellation, streaming logs
Queue Small explicit single-consumer actor One owner for sync and profile operations
Logging Fastify/Pino structured logs Client ID, request ID, operation ID, sync phase
Tokens Isolated auth service with replaceable token repository Token format remains an open decision
MVP state Validated JSON repository with atomic file replacement Matches current product decision
Later state SQLite repository behind the same interfaces Grants, operations, cursors, subscriptions
Testing Built-in Node test runner or Vitest; adapter contract tests Fast unit tests plus real CLI integration suite

Node's built-in SQLite module is currently documented as release-candidate stability rather than fully stable, so the project should not commit to it yet. The storage interface allows later selection among node:sqlite, a mature SQLite binding, or a query builder when SQLite work begins. See Node SQLite.

Architecture Rules Independent of Language

One process owns the profile

The service must run as one application instance with one profile actor. Do not enable:

  • Node cluster mode;
  • multiple Uvicorn/Gunicorn workers;
  • multiple container replicas;
  • multiple JVM/.NET web workers that each believe they own the profile.

HTTP concurrency is allowed; Joplin profile concurrency is not. Requests queue at the application-service boundary.

REST and GraphQL contain no Joplin orchestration

Both transports call shared use cases such as:

  • ListVisibleNotes;
  • CreateNote;
  • RestoreRevision;
  • PollVisibleChanges.

Those use cases perform authorization and submit one operation to the profile actor. HTTP handlers and GraphQL resolvers must never invoke Joplin directly.

The profile actor owns the complete operation

One queued operation contains:

  1. pre-sync;
  2. hierarchy refresh and authorization;
  3. local read or mutation;
  4. event/operation recording;
  5. post-sync for mutations;
  6. final or degraded result.

The lock must not be released between these phases.

Events use an outbox boundary

Future WebSocket delivery should consume recorded gateway events rather than receiving ad hoc resolver notifications. Even if the MVP event repository is minimal, event IDs and envelopes should be created in the application layer so delivery transports can change later.

Configuration and generated state use separate repositories

The administrator JSON repository is read-only after process startup. The generated-state repository owns automatic notebook grants and later operation/event state. Code must not assume both are one file merely because both use JSON in MVP.

Joplin is never accessed through its SQLite database directly

Reading or writing Joplin's database would couple the gateway to private schema, bypass application invariants, and conflict with profile ownership. The gateway's future SQLite database is separate gateway state.

Required Technical Spike

Before creating the full service, build a disposable adapter experiment with a fresh test profile and non-production sync target.

Questions to answer

  1. Can the CLI local API server and explicit sync operate in one process/profile without concurrent access?
  2. Does the server process perform recurring sync in headless command mode, and can completion be observed reliably?
  3. Can the gateway trigger and await sync without stopping the Data API server?
  4. If stop/sync/start is required, what are the latency and failure characteristics?
  5. Can all MVP note, notebook, tag, trash, search, revision, and event operations be implemented through structured APIs?
  6. Can a historical revision be reconstructed and restored to an explicitly chosen notebook without direct database access?
  7. What outputs and exit codes distinguish authentication, network, conflict, validation, and profile-lock failures?
  8. What happens if the Joplin process is killed during each sync or mutation phase?
  9. How does E2EE initialization/unlocking behave in an unattended profile?
  10. Which behaviours change across the oldest and newest Joplin CLI versions the gateway intends to support?

Spike exit criteria

The spike should produce:

  • a short adapter decision record;
  • executable tests for the required flows;
  • captured structured outputs and error cases;
  • measured cold-start, sync, and request latency;
  • a minimum supported Joplin CLI version;
  • a clear selection among one-shot commands, local Data API, direct libraries, or helper process.

Only then should the language recommendation be frozen.

Final Recommendation

Proceed with a TypeScript adapter spike because it can test every integration route, including direct Joplin packages if necessary. Keep the spike behind a JoplinAdapter interface and do not build HTTP endpoints yet.

If documented CLI/local-HTTP integration proves complete, retain TypeScript unless the project has a strong preference for Go. TypeScript still provides the simplest container and fastest REST/GraphQL delivery.

If direct Joplin internals are required, TypeScript becomes the practical choice, but the project must consciously accept version coupling and resolve AGPL compliance.

If a separate Joplin helper becomes necessary, reassess TypeScript versus Go for the public gateway. In that architecture, Go plus gqlgen becomes a particularly strong option because Joplin coupling has moved behind a private process boundary.

Rust, C#, Kotlin/Java, Python, and Elixir can all implement the system correctly. None currently offers enough project-specific advantage to displace the TypeScript/Go shortlist.