TypeScript codegen backend

[cometkim]

See #6196 (comment) for context.

I tried to implement all the features we need, including those not supported by genType today.

Currently, it is somewhat complex to support both .ts and .d.ts outputs. Eventually, it could simplify the codegen process by unifying the codebase after deprecating genType and by organizing the context and config structures.

Config

For dts:

{
  "sources": "src",
  "package-specs": [
    {
      "module": "esmodule",
      "in-source": true,
+     "dts": true
    }
  ]
}

For TypeScript

{
  "sources": "src",
  "package-specs": [
    {
-     "module": "esmodule",
+     "module": "typescript",
      "in-source": true
-     "dts": true
    }
  ]
}

Attributes

• @as: Rename the generated types.

  type user // => interface user
  
  @as("User")
  type user // => interface User

• @external("TypeName"): Use external (global) type

• @external(("package-name", "Name", true)): named type import

• @external(("package-name", "default", true)): default import as type

• @external(("package-name", "*", true)): namespace import as type

• @opaque: Force the type to be opaque

  type t 
  // abstract types will automatically be opaque
  
  type t<'a, 'b> = 'b
  // opaque type because 'a is phantom type
  
  @opaque
  type t = string
  // branded string type

[pkg-pr-new]

Open in StackBlitz

rescript

npm i https://pkg.pr.new/rescript-lang/rescript@8118

@rescript/darwin-arm64

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/darwin-arm64@8118

@rescript/darwin-x64

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/darwin-x64@8118

@rescript/linux-arm64

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/linux-arm64@8118

@rescript/linux-x64

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/linux-x64@8118

@rescript/runtime

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/runtime@8118

@rescript/win32-x64

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/win32-x64@8118

commit: d997889

[cometkim]

format indentation

[cometkim]

Not sure this is really useful. they are value-only types.

[nojaf]

Hi there,

I played with this in rescript-kaplay and when adding the "dts": true, it created a bunch of .d.mts files.
In those files I noticed that namespaces still have hyphens:

import type * as rescript from "@rescript/runtime/types";
import type * as Color-Kaplay from "@nojaf/rescript-kaplay/src/Components/Color.res.mjs";
import type * as Types-Kaplay from "@nojaf/rescript-kaplay/src/Types.res.mjs";
import type * as Vec2-Kaplay from "@nojaf/rescript-kaplay/src/Vec2.res.mjs";

export type t = rescript.opaque<"Wall-Skirmish.t", []>;

export interface rectOptions {
  readonly radius?: rescript.option<number>;
  readonly fill?: rescript.option<boolean>;
}

export interface areaCompOptions {
  readonly shape?: rescript.option<Types-Kaplay.shape<Vec2-Kaplay.Local.t>>;
  readonly offset?: rescript.option<Vec2-Kaplay.Local.t>;
  readonly scale?: rescript.option<number>;
}

in that project I do a lot of module inclusion stuff like:

include GameObjRaw.Comp({type t = t})
include Pos.Comp({type t = t})
include Sprite.Comp({type t = t})
include Area.Comp({type t = t})

(sample)

the generation there isn't quite right for the game objects.

When I tried "language": "typescript", it looked mostly fine expect for all the namespaces.

Just echoing some feedback here, keep it up!

[cometkim]

@nojaf thanks.

I found two problems in the project you shared:

1. package namespace
2. nested record definitions

Both are trivial to fix. And I don't see any other significant issues with the generated type definitions.

Let me fix them.

[cometkim]

I would make the typescript output as another package-spec. the final module compilation will be handled by tsc anyway.

[zth]

@cometkim looks fantastic! Awesome work!

I tried it on a few things, one was res-x: https://github.com/zth/res-x

It surfaces a few issues that's probably easier to chase down by just trying it on that repo. Some examples:

Namespaces seems to not be included in some references

A namespace for Actions is defined with a type action:

module Actions: {
  type t

  @tag("kind")
  type target = This | CssSelector({selector: string})

  @tag("kind")
  type rec action =
    | ToggleClass({target: target, className: string})
    | RemoveClass({target: target, className: string})
    | AddClass({target: target, className: string})
    | SwapClass({target: target, fromClassName: string, toClassName: string})
    | RemoveElement({target: target})
    | CopyToClipboard({
        text: string,
        onAfterSuccess?: array<action>,
        onAfterFailure?: array<action>,
      })

  let make: array<action> => t
} = {
  type t = string

  @tag("kind")
  type target = This | CssSelector({selector: string})

  @tag("kind")
  type rec action =
    | ToggleClass({target: target, className: string})
    | RemoveClass({target: target, className: string})
    | AddClass({target: target, className: string})
    | SwapClass({target: target, fromClassName: string, toClassName: string})
    | RemoveElement({target: target})
    | CopyToClipboard({
        text: string,
        onAfterSuccess?: array<action>,
        onAfterFailure?: array<action>,
      })

  external stringifyActions: array<action> => string = "JSON.stringify"

  let make = actions => stringifyActions(actions)
}

Generates:

declare namespace Actions {
  type t = rescript.opaque<"ResX.Client.Actions.t", []>;
  type target =
    | "This"
    | { readonly kind: "CssSelector"; readonly selector: string };
  type action =
    | {
  readonly kind: "ToggleClass";
  readonly target: Actions.target;
  readonly className: string;
}
    | {
  readonly kind: "RemoveClass";
  readonly target: Actions.target;
  readonly className: string;
}
    | {
  readonly kind: "AddClass";
  readonly target: Actions.target;
  readonly className: string;
}
    | {
  readonly kind: "SwapClass";
  readonly target: Actions.target;
  readonly fromClassName: string;
  readonly toClassName: string;
}
    | { readonly kind: "RemoveElement"; readonly target: Actions.target }
    | {
  readonly kind: "CopyToClipboard";
  readonly text: string;
  readonly onAfterSuccess?: rescript.option<Actions.action[]>;
  readonly onAfterFailure?: rescript.option<Actions.action[]>;
};
}
export type Actions = {
  make: (arg0: Actions.action[]) => Actions.t;
};

// --NOTICE-- here how `actions` is not referenced from the namespace
function make(actions: action[]): string {
  return JSON.stringify(actions);
}

let Actions: Actions = {
  make: make
};

Some packages tries to import from @rescript/runtime

res-x depends on rescript-bun. But, it seems references to rescript-bun is being imported from @rescript/runtime:

import type * as Globals$RescriptBun from "@rescript/runtime/lib/es6/Globals$RescriptBun.js";

export type URLSearchParams = {
  copy: (arg0: Globals$RescriptBun.URLSearchParams.t) => Globals$RescriptBun.URLSearchParams.t;
};

rescript-bun ships not @external or anything, so it's all plain (although that would be the next thing I check, if @external propagates from packages, so they can refer to real TS types via that as well).

Invalid code sometimes generated

This code:

let files = await loadStaticFiles()
    let files =
      files
      ->Array.map(f => {
        (
          switch isDev {
          | true if f->String.startsWith("public/") => f->String.slice(~start=7)
          | false if f->String.startsWith("dist/") => f->String.slice(~start=5)
          | _ => f
          },
          f,
        )
      })
      ->Map.fromArray

Generates syntax invalid code:

let files = await loadStaticFiles(undefined);
    let files$1 = new Map(files.map(f: string: [string, string] => [
      isDev ? (
          f.startsWith("public/") ? f.slice(7) : f
        ) : (
          f.startsWith("dist/") ? f.slice(5) : f
        ),
      f
    ]));

Note the type annotations in files.map.

[cometkim]

@zth @nojaf I've fixed all the issues you mentioned. Would you like to try again?

[nojaf]

The namespace issue got resolved, but the include module parts aren't quite accurate.

As mentioned in the docs.
GameObjects typically look like:

module Hero = {
  type t = { name: string }

  open Kaplay

  // We want to draw an image on the screen for an object
  include Sprite.Comp({type t = t})
  // We want to change the position of the object
  include Pos.Comp({type t = t})
}

And a component looks like:

module Comp = (
  T: {
    type t
  },
) => {
  @send
  external numFrames: T.t => int = "numFrames"

  @send
  external play: (T.t, string) => unit = "play"

  @get
  external getSprite: T.t => string = "sprite"

  @set
  external setSprite: (T.t, string) => unit = "sprite"

// ...
}

(sample)

So I would expect the interface of Hero.t to be:

{ name: string } + everything it got from the include modules.

Right now I see:

export interface t {
  readonly name: string;
}

export interface spriteCompOptions {
  readonly frame?: rescript.option<number>;
  readonly width?: rescript.option<number>;
  readonly height?: rescript.option<number>;
  readonly anim?: rescript.option<string>;
  readonly singular?: rescript.option<boolean>;
  readonly flipX?: rescript.option<boolean>;
  readonly flipY?: rescript.option<boolean>;
}

I understand this is probably quite the edge-case, but well, you asked 😉.

[cometkim]

That's expected one. external has no outputs for either js and dts

[cometkim]

Status update: d.ts is almost complete. However, the TypeScript output is getting messier.

I don't expect this to add too much complexity. It should be possible to implement it with a reasonable level of complexity.

Ideally, we could make it a single, unified process based on the new TS IR, with distinct printers. Omitting the type annotation produces the original JavaScript output, and omitting the implementation body produces the d.ts output. That's it.

Right now, I'm avoiding modifying the existing code, which makes it feel very fragmented and unnecessarily complex. It's not only difficult to understand but also inefficient.

I'm about to embark on a massive refactoring phase. This could create significant conflicts with new PRs, but I believe it's necessary.

[cknitt]

Tested against one of our company projects.

It seems that type definitions for types that are private to a module (not exported in .resi) are not emitted to .ts, but those types are still referenced in function signatures etc. in the .ts output.

[cknitt]

In ReScript, I can do

let map = Map.make()

and the type of map will be inferred correctly based on later usage. But this is not the case for TypeScript. The .ts output for the above is currently

let schemas = new Map();

which has type Map<any, any>.

We would need to emit something like

let schemas: Map<string, number> = new Map();

(or whatever the concrete types are in the given case) to get rid of the any here.

[cknitt]

Also saw that one:

Some packages tries to import from @rescript/runtime