Retrying module import from other folders

[obones]

I have mono repo with a structure very similar to this:

Common
   |- Feature1
   |    \- index.js
   |- someUtils.js
   \- someOtherUtils.js
Lambda1
   |- index.js
   \- package.json
Lambda2
   |- index.js
   \- package.json
test
   |- Common
   |    |- Feature1
   |    |    |- index.js
   |    |    \- package.json
   |    |- package.json
   |    \- someUtils.js
   |- Lamda1
   |    |- index.js
   |    \- package.json
   \- Lamda2
        |- index.js
        \- package.json

The CI system creates the zip files that are deployed on the AWS Lambda system and does so by taking the content of lambda folder and placing next to it the content of the Common folder renamed as node_modules.
This has worked since the beginning and when unit tests were added later on, it was decided to have as little changes to the existing code as possible.
Each .js file inside the test folder uses module-alias to trick the loader into looking at the Common folder as a valid source for the files required by the lambda code. This allows debugging the unit tests while being able to edit the common files "in place", greatly speeding up the development process.
Here is the code added at the top of each .js file in the test subfolders:

const module_alias = require("module-alias");
module_alias.addPath(__dirname + "/../../Common");
module_alias.addPath(__dirname + "/node_modules/");

The second path alias is here because only the package.json files in the test folders are defining the devDependencies to @aws modules because they already are available in the target environment, and because the entry point is via tests only. This avoids cluttering the developer's machines with lots of duplicate folders.

This works fairly well in CommonJS format but as we must modernize the code base, we are trying to move to ES6 syntax and "module" resolution with as little changes as possible.

One thing that is clear is that adding path aliases like before is no longer possible because imports are resolved long before any of our code is run. This has to be done via the --import flag on the command line so that I can register a module hook which is what I tried with the following code, inspired by esm-module-alias:

import { register } from "node:module";
import { fileURLToPath } from "node:url";
import path from "node:path";

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

register("./loader.mjs", import.meta.url);

const windowsSupportString = process.platform === 'win32' ? 'file://' : '';
const base = removePrefix(__dirname, windowsSupportString); 

function removePrefix(value, prefix) {
   return value.startsWith(prefix) ? value.slice(prefix.length) : value;
}

function generateResolver() {
  return async (specifier, parentModuleURL, nextResolve) => {
    try
    {
        return await nextResolve(specifier, parentModuleURL);
    }
    catch (error)
    {
        if (error?.code === 'ERR_MODULE_NOT_FOUND')
        {
            let retryParentModuleURL = Object.assign({}, parentModuleURL);
            let retrySpecifier = specifier;

            retryParentModuleURL.parentURL = windowsSupportString + path.join(base, "..", "Common");

            try
            {
                console.log(`RETRY: ${retrySpecifier} from ${JSON.stringify(retryParentModuleURL)}`);
                return await nextResolve(retrySpecifier, retryParentModuleURL);
            }
            catch (innerError)
            {
                console.log("/INNER ------------");
                console.log(innerError);
                console.log(JSON.stringify({retrySpecifier, retryParentModuleURL}));
                console.log("/INNER ------------");
                throw innerError;
            }
        }

        throw error;
    }
  };
}

export const resolve = generateResolver();

This is not exactly the same as the situation before because the loader.mjs file is loaded once at the start of the process called by the test runner (mocha) which means __dirname is the one from that file, not the one for each test file.
But even then, the retry appears to work as I see the console log, but sadly it still throws an error, but this time with a hint message:

Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@aws-sdk/lib-dynamodb' imported from C:\Sources\myproject\src\api\test\Common
Did you mean to import "@aws-sdk/lib-dynamodb/dist-cjs/index.js"?

It's like it has "seen" the folder but did not find the expected files for ES modules, which I find weird as they appear to be in the dist-es folder.

Am I on the right track here? Would you have any suggestions for this situation?

I looked at subpath imports (with the # prefix) but this does not work because it cannot look into parent folders for resolution.

Thanks for your time.

[ljharb]

I'm confused - why not just take the actual project folder, run npm prune --production, and tar up everything that's left?

[obones]

Because that's a monorepo where each "Lambda" folder has its own pack script that creates a zip file. There must be one zip file per lambda as this is what's required by the AWS tools to deploy a lambda.

[ljharb]

ok, but you can still do that in each folder by running npm install --no-workspaces --production directly in each subfolder.

[obones]

But the main issue here is not really creating the deployment packages but rather have the IDE (VSCode with MochaTestExplorer) be able to find the dependencies and use them "in place" so that it's easier to debug and change the common files if need be.