Good enough type safety for my API with deno and oak
As a hobby project I develop a Telegram messenger for old Java ME feature phones, unimaginatively called Microgram. That app relies on a server application that works as a proxy between Telegram and Microgram. That proxy server is developed with deno and oak.
Naturally, I want to validate the API requests and work with safely typed data. And I don't want to dive deep into the fascinating universe of JSON validation, I just want to have something good enough. I thought this is a very common problem, but the only built in feature that I found was context assert. This is not nothing, , but this leaves a lot to do. The pragmatic solution would be to use something like AJV and TypeBox. But since this is a hobby project and I can do what I want, I want to shave this yak and come up with a "good enough" solution on my own.
My first attempt (don't do this)
My first attempt was just one step further from the "get started" tutorial from oak: I moved my routes into a separate file, listed the names of the parameters, specify the functions and build a small wrapper that would call the function with the listed POST or GET parameters as JSON object. A typical route would look like this:
//routes.ts
[
{
name: '/chats',
method: 'get',
params: ['sessionString', 'number', 'limit'],
func: getChats,
},
…
]func (in this case getChats) being a function of this type: (ctx: Context, params: Record<string, string | number | boolean>) => Promise<void> | void.
On application start, I iterate over my routes and add them to oak's Application, each with a function that gets the values with the route's params keys from the request and then calling the route's func with it. I just go through the route's parameter keys and create an object with the request's data. getChats then gets an object with whatever we got in the request under they keys sessionString, number and limit.
The obvious problems:
- No validation (getChats on its own has to find out what it got)
- No possibility for obligatory and optional parameters (meaning that everything is optional)
- No type safety, the called function just gets a generic object as
Record<string, any>
What I do now instead
First, the parameter of getChats should get a proper type.
declare type GetChatParams = {
sessionString: string,
number: string,
limit?: number,
};
function getChats(ctx: Context, GetChatsParams params) {
// process parameters, query Telegram, send response via ctx…
}To transform evil input from the internet to a safe and typed GetChatsParams, we have to implement some validation functions. These should do more than just a typeof value === 'string'. Because we know pretty good how e.g. a Session String from Telegram looks like, we can be more specific:
// validator.ts
export const validator = {
// number has valid format for Telegram (or is a Telegram test number 99966…)
isTelNumber: (s: any, optional = false) => opt(s, optional) || (typeof s === 'string' && /^\+[0-9]{5,15}$/.test(s) || /^99966\d{5}$/.test(s)),
isPositiveInteger: (n: any, optional = false) => opt(n, optional) || (Number.isInteger(n) && n >= 0),
isString: (s: any, optional = false) => opt(s, optional) || typeof s === 'string',
isNonEmptyString: (s: any, optional = false) => opt(s, optional) || (typeof s === 'string' && s.length > 0),
// session string has a plausible length and ends with "="
isSessionString: (s: any, optional = false) => opt(s, optional) || (typeof s === 'string' && s.length > 300 && s.length < 500 && s.endsWith('=')),
isBigIntegerAsString: (s: any, optional = false) => opt(s, optional) || (typeof s === 'string' && /^-?\d+$/.test(s)),
};As you see, for session strings or telephone numbers, positive integers or non-empty strings, we do more than just a basic type check.
Now in the route's definition, we split the parameter into obligatory and optional. Also, we assign one of the validator function names to all of them:
// routes.ts
[
{
name: '/chats',
method: 'get',
params: {
number: 'isTelNumber',
sessionString: 'isSessionString',
},
optionalParams: {
limit: 'isPositiveInteger',
},
func: getChats,
},
…
]Now in application startup, we can iterate over the routes array again and register them, but now we can go over the params and optionalParams and call the appropriate function of the validator for each parameter like this:
Object.keys(params).forEach(key => {
const validationFunction = validator[params[key]];
const data = requestData[key];
if (!validationFunction(data))
// fail with error message
});Anyhow. After running the validation functions, we can be certain that the request data complies to our defined restrictions. But now we run into a typing problem: The func property of the Route type has the signature (... params: Params) => void, with Params being a union of all parameter types of all endpoints. But getChats only accepts 1 of those types, GetChatsParams and not the parameter types of other endpoints, e.g. SendMessageParams.
The solution: func is not directly getChats, but a small, inline helper function that performs the necessary cast.
{
name: '/chats'
…
func: (c, params) => getChats(c, params as GetChatsParams),
}And now we can easily define our routes and have it all type safe and validated without those abstractions of off-the-shelf schema validators and runtime type systems.
Well, here you have my take on the problem of API validation. What I like about my approach is that it handles GET and POST requests in the same way. What I don't like is that types are basically defined twice, once as a real type and once in the route definition with the validator functions. As I said, it is good enough for me and I can proceed with other aspects of the software. If you have any comments on this, I would be happy to hear from you.
To wrap it up, here comes the final code:
// validator.ts
import { Context } from "https://deno.land/x/oak@v11.1.0/mod.ts";
const opt = (value: any, optional: boolean) => !optional || (optional && typeof value === 'undefined');
export const validator: Validator = {
// number has valid format for Telegram (or is a Telegram test number 99966…)
isTelNumber: (s: any, optional = false) => opt(s, optional) || (typeof s === 'string' && /^\+[0-9]{5,15}$/.test(s) || /^99966\d{5}$/.test(s)),
isPositiveInteger: (n: any, optional = false) => opt(n, optional) || (Number.isInteger(n) && n >= 0),
isString: (s: any, optional = false) => opt(s, optional) || typeof s === 'string',
isNonEmptyString: (s: any, optional = false) => opt(s, optional) || (typeof s === 'string' && s.length > 0),
// session string has a plausible length and ends with "="
isSessionString: (s: any, optional = false) => opt(s, optional) || (typeof s === 'string' && s.length > 300 && s.length < 500 && s.endsWith('=')),
isBigIntegerAsString: (s: any, optional = false) => opt(s, optional) || (typeof s === 'string' && /^-?\d+$/.test(s)),
};
export function validateParams(params: Record<string, keyof Validator>, values: Record<string, any>, optional: boolean) {
const validatedParams: Params = {};
Object.keys(params).forEach(key => {
const validateFunction = validator[params[key]];
if (!validateFunction(values[key], optional)) {
throw new Error('Invalid value for ' + key);
}
if (values[key]) {
validatedParams[key] = values[key];
}
});
return validatedParams;
}
export type Validator = {
isTelNumber: (s: any, optional: boolean) => boolean
isPositiveInteger: (s: any, optional: boolean) => boolean
isNonEmptyString: (s: any, optional: boolean) => boolean
isSessionString: (s: any, optional: boolean) => boolean
isString: (s: any, optional: boolean) => boolean
isBigIntegerAsString: (s: any, optional: boolean) => boolean
};
export type Validation = keyof Validator;
// routes.ts
declare type Route = {
name: string,
method: 'get' | 'post',
params?: Record<string, Validation>,
optionalParams?: Record<string, Validation>,
func: (
ctx: Context,
server: MicrogramServer,
params: Params,
) => Promise<void> | void,
};
declare type EmptyParams = Record<string | number | symbol, never>;
declare type AuthParams = {
sessionString: string,
number: TelephoneNumber,
};
declare type GetChatParams = AuthParams & {
limit?: number,
};
declare type Params = GetChatParams | EmptyParams;
const routes: Route[] = [
{
name: '/chats',
method: 'get',
params: {
number: 'isTelNumber',
sessionString: 'isSessionString',
},
optionalParams: {
limit: 'isPositiveInteger',
},
func: (c, s, params) => getChats(c, s, params as GetChatParams),
}
]And finally the oak Application:
// server.ts
import Routes from "./routes.ts";
import { Application, Router } from "https://deno.land/x/oak@v11.1.0/mod.ts";
const app = new Application();
const router = new Router();
Routes.forEach((r: Route) => {
switch (r.method) {
case 'get':
router.get(r.name, (ctx: Context) => this.microgramMiddleware(ctx, r));
break;
case 'post':
router.post(r.name, (ctx: Context) => this.microgramMiddleware(ctx, r));
break;
}
});
private async microgramMiddleware(ctx: Context, r: Route) {
try {
let combinedParams: Params = {};
if (r.params) {
const obligatoryValues = await this.getReqValues(ctx, r.method, r.params);
combinedParams = validateParams(r.params, obligatoryValues, false);
}
if (r.optionalParams) {
const optionalValues = await this.getReqValues(ctx, r.method, r.optionalParams);
validateParams(r.optionalParams, optionalValues, false);
combinedParams = {...(combinedParams || {}), ...optionalValues};
}
await r.func(ctx, this, combinedParams);
} catch (e) {
// respond with e.message
}
}