Using Easy-Schema for OpenAPI doc generation?

[ceigey]

This might get out of scope fast.

I'm in the early stages of a migration from Meteor to NestJS, as my team shifts towards an API-centric architecture. Two key selling points of NestJS - can attach it to an existing Express instance (e.g. Meteor's webapp package), and the OpenAPI doc generation from the same DTOs that are used for validation.

But I was just thinking that this could be made idiomatic for Meteor with the tools available. Your package jam:easy-schema is already able to generate JSON Schema for MongoDB (draft 4), which is a big chunk of the work IMO.

OpenAPI versioning is a bit messy but 3.1 appears to be using draft 7. There's a breaking change in there but it seems to be pretty much id -> $id and some semantics for numerical ranges.

The tricky part is how to integrate this into the ecosystem.

simple:rest and meteor-rest are interesting examples of Method & Publication reuse, but perhaps that's out of scope and would require modifications to things like jam:method.

The simplest (to implement, and reason about) approach I can think of is essentially wrapping WebApp.handlers.... or an express router with a function that records the route path, method, metadata, etc, and builds it all into an OpenAPI spec.

Something like:

import { EasySchemaRouter } from '....'

const router = new EasySchemaRouter()

// Easy Schema stuff goes in here:
router.post(
  '/send-message-to/user/:userId',
  {
    params: { userId: String },
    body: { text: String, priority: Number }
  },
  async (req, res) =>  { /* expressy stuff here */ }
)

Then later on:

WebApp.handlers.use(router)
WebApp.handlers.use('/api/openapi', /* Scalar or Swagger */)
WebApp.handlers.use('/api/openapi.json', router.openApiDocs({ /* ... config goes here ... */ })

Some additional thoughts:

• I don't think Easy Schema has coercion. So that would have to be explored if required e.g. for params, especially if this went to another step and involved reusing jam:method definitions for API endpoints somehow (too far ahead for me to imagine how to integrate that)
• Speaking of jam:method, I believe that also supports Zod, so ideally whatever solution this might evolve into should be aligned and accept the same options.
• Eventually something like simple:rest IMO is the ideal, but with more configuration on the method side. E.g. being able to dictate globally how paths should be split up, being able to mark certain methods as GET, PATCH, PUT, being able to give a method a HTTP path that has some path parameters and then transform the path parameters and json body into whatever format your method parameters normally come in, etc.

[jamauro]

Sounds really interesting. Will think on it a bit. I think we should start with the ideal and work backwards.

[ceigey]

Sounds good to me!

[ceigey]

I'm having to refresh my memory since I only touched on simple-rest for a split second before never really getting to take advantage of it, but it looks like e.g. for ValidatedMethod they avoided monkey-patching by using mixins:

simple:rest-method-mixin

const method = new ValidatedMethod({
  name: 'method',
  mixins: [RestMethodMixin],
  validate: ...,
  restOptions: {
    url: '/my-custom-url',
    // any other options
  },
  run() {
    return 5;
  }
});

Alas, the important part is inside that:

https://github.com/stubailo/meteor-rest/blob/devel/packages/rest-method-mixin/rest-method-mixin.js

SimpleRest.setMethodOptions(options.name, options.restOptions);

... which is used for monkey patching Meteor.methods.

https://github.com/stubailo/meteor-rest/blob/devel/packages/rest/rest.js#L99

Meteor.method = function (name, handler, options) {
  if (!SimpleRest._methodOptions[name]) {
    SimpleRest.setMethodOptions(name, options);
  } else if (options) {
    throw Error('Options already passed via setMethodOptions.');
  }
 // ... more stuff here
}

The //... more stuff here part is specifically inferring POST PATCH REMOVE by convention for Collection methods, e.g.

var insideDefineMutationMethods = false;
var oldDMM = Mongo.Collection.prototype._defineMutationMethods;
Mongo.Collection.prototype._defineMutationMethods = function () {
  insideDefineMutationMethods = true;
  oldDMM.apply(this, arguments);
  insideDefineMutationMethods = false;
};

(I think that might be a step too far and it's probably better to just focus on user-declared methods and pub/sub and make things opt-in, otherwise forking simple-rest and getting it to work with Meteor v3 + easy-schema might be simpler)

So basically it seems to work how I could imagine, just wrap calls and before registering the method make sure you register the API endpoint + set up all the OpenAPI stuff.

Some problems I still haven't found solutions for:

• Meteor.user and Meteor.userId seem to historically be flakey or have other issues with simple-rest, and patching them pre and post request might not be wise (monkey-patching globally and using express middelware with AsyncLocalStorage might work though)
• Explicitly setting other properties like the HTTP method and path can't be done with monkey patching Meteor.methods (but it could be done with jam:method or ValidatedMethod because of the object properties and mixins).

Using jam:method and jam:pub-sub as inspiration, maybe some clever use of adding new properties to globals or otherwise riffing on existing APIs could be used to make it easy to migrate an existing codebase and keep it explicit which code runs in which environment, for example:

import { Rest } from 'something:rest'

// or Meteor.methods.rest({ ... })
Rest.methods({
  // normal methods just assume post semantics by default
  async 'user.incrementViewCount'() { .... },

  // wrap in object to add other config
  'user.profile.update': {
    /* method: 'PATCH',
    path: '/user/:id/profile', */
    // or use Golang style paths
    path: 'PATCH /user/:id/profile',
    // or infer from path: https://dev.to/bytimo/useful-types-extract-route-params-with-typescript-5fhn
    params: { id: String },
    // not required with jam:method at least
    body: { name: Match.optional(String) },
    /* responses: {
      '200': { id: String, name: String },
    }, */
    // or just infer for simplicity's sake?
    response: {
      '200': { id: String, name: String },
    },
    remixInputs: (params, body) => ({ id: Number(params.id), ...body }),
    // I'm just stealing this from ValidatedMethod and jam:method 
    async run({ id, name }) {
      const user = await Rest.user()
      if (user != ...) { ... }
      // some DB operations
      return profile
    }
  }
})

Rest.publish({ ... })

I'm seeing a lot of opportunity to make this modular enough to extend/configure for different registration strategies of methods, and same for different kinds of schemas and reduce the coupling for easy-schema etc.

E.g.

Meteor.methods.rest.patchConfig({
  schemaToOpenAPI(schema) {
    if (isValibotSchema(schema)) { ... }
    if (isJamSchema(schema)) { ... }
  },
  checkSchema(schema) {
    if (isJamSchema(schema)) { ... }
    // etc
  },
  // Oh no, did webapp get updated and it's now Hono under the hood and the maintainer's asleep?
  registerEndpoint(options: RestEndpointOptions /* see above example */) {
    WebApp.handlers.on(options.method, options.normalizedPath, ...myManualReplacementMiddlewares, ... /* etc */)
  },
})

declare 'meteor/something:rest' {
  // somehow merge a declaration for handling an arbitrary schema here
}

This comment got out of hand, so I might go away and prototype something like ceigey:rest.

After that, it might be a discussion of what things should be contributed directly to easy-schema (vs potential niche bloat), and what's a reasonable way to interop with everything.

I'm imagining an API like:

• Rest.methods, wrapping Meteor.methods
• Rest.user, wrapping Meteor.user with AsyncLocalStorage
• Rest.userId, wrapping Meteor.userId with AsyncLocalStorage
• Rest.createMethod, wrapping jam:method's createMethod
• Rest.publish, wrapping Meteor.publish
• Rest.publish.once, wrapping jam:pub-sub's Meteor.publish.once

[jamauro]

I think that might be a step too far and it's probably better to just focus on user-declared methods and pub/sub and make things opt-in

Agree

// normal methods just assume post semantics by default

I was thinking the same

Meteor.user and Meteor.userId seem to historically be flakey or have other issues with simple-rest, and patching them pre and post request might not be wise (monkey-patching globally and using express middelware with AsyncLocalStorage might work though)

I think there are ways around this but I'd have to try them out :) One thought would be to get the user info via middleware and set the DDP context so its available when the method is invoked.

--

Overall, I think you're definitely on to something valuable and you've clearly given it more thought than I have. I wonder what pieces we could simplify so people wouldn't need to necessarily learn new semantics / syntax.

I took a quick look at simple:rest and I suppose we could add more options to jam:method so you define things inside createMethod but I think I prefer the idea of having all my api stuff in one spot so I can quickly scan what's available.

I haven't given it too much thought, but I think something like this could be pretty nice:

import { api } from 'package';
import { someMethod } from 'somewhere';

// defaults, zero config needed
api.configure({ 
  path: '/api', 
  documentation: '/openapi.json',
  // these could be used to provide something similar to the simple:rest philosophy 
  // not sure if it's a good idea or not but if someone wanted to be kind of lazy and not explicitly identify everything :)
  exposeAllMethods: false, // when true, methods are available at /api/{methodname} and use POST
  exposeAllPublications: false // when true, publications are available at /api/{publicationname} and use GET
})

// these are wrapping WebApp. the idea would be to make it very familiar / similar to express but much easier :)

// I think we could grab the schema from `jam:method` with a little magic and generate its open api docs. for others, I suppose it'd require passing something in
// could all the other info be inferred as well? It'd be nice to not have to think about any of it really :)
api.post(path, someMethod); // or magic method string name?
api.get(path, somePublication) // or magic publication string name? somePublication I guess would necessitate createPublication existing similar to createMethod

// define custom ones
api.post(path, handler); 
api.get(path, handler); 

[ceigey]

One thought would be to get the user info via middleware and set the DDP context so its available when the method is invoked

That'd be good, certainly the less search-and-replaces would be beneficial.

I like the look of your example too. I was thinking "well if we're just wrapping express a little to add schemas then we could just as well tell people 'just use Nest in Meteor using this one simple trick'" but I like the idea of explicitly wrapping an already declared resource.

I think a tricky thing is native Meteor methods/publications, because we probably don't want the schema redeclared just for OpenAPI purposes (e.g. if the developer is using check in the body of their function. I guess that'd require people to use jam:method to avoid duplicating type checks.