October release - 2020
Alright! Here we go with another release of AdonisJS. This is a big one in terms of the development workflow you will be using moving forward.
Highlights
- Add support for in-memory compilation of TypeScript. In other words, the compiled JavaScript is not written to the disk anymore.
- Introduce @adonisjs/repl to quickly test out your application code.
- Add support for validating Environment variables .
- Add support for conditional query constraints using the
if
,unless
, and thematch
helpers.
Steps to upgrade
Before we dive into the specifics and the motivation behind the changes. Let's quickly talk about the steps you will have to take to upgrade your application.
-
We have recently encapsulated a lot of eco-system dependencies within the @adonisjs/core package and hence those dependencies can be removed from your project.
Run the following command to remove
@adonisjs/fold
and@adonisjs/ace
.npm uninstall @adonisjs/fold @adonisjs/ace -
Next, upgrade all dependencies to their latest alpha version. Do remember to install with the
@alpha
tag. -
Now since your application is not using
@adonisjs/ace
as a direct dependency, you must update your local commands inside thecommands
directory to use@adonisjs/core
for importing theBaseCommand
.- import { BaseCommand } from '@adonisjs/ace'+ import { BaseCommand } from '@adonisjs/core/build/standalone' -
The commands
handle
method has been deprecated in favor of the therun
method. It feels natural to say run the command vs saying handle the command. -
The process started by the command will close itself after the
run
method has finished executing. If you want your commands to stay alive, then you need to use thestayAlive
flag on the settings object.class MyCommand extends BaseCommand {public static settings = {stayAlive: true,}} -
Also update the
./commands/index.ts
file to use the following code snippet.import { listDirectoryFiles } from '@adonisjs/core/build/standalone'import Application from '@ioc:Adonis/Core/Application'export default listDirectoryFiles(__dirname, Application.appRoot, ['./commands/index']) -
If you are using the
@adonisjs/auth
package. You need to update theconfig/auth.ts
file to lazy import the models used for finding the users.So begin by removing the top level import statement
config/auth.tsimport User from 'App/Models/User'And move it inline next to the model property as follows:
provider: {driver: 'lucid',model: () => import('App/Models/User')} -
Finally, we have deprecated the
Env.getOrFail
method in favor of the Env validations. You just need to find its usages and replace it withEnv.get
to avoid getting deprecation warnings. -
Using Japa as your test runner? Here are the instructions to upgrade the test runner to run tests using the TypeScript source directly.
In-memory TypeScript compilation
I am not a big fan of build tools or adding an additional step to prepare my code for getting executed. However, when using TypeScript there is no way to escape the process of compiling TypeScript to JavaScript since v8 is meant to run JavaScript only.
Initially, we did add a build step in which we compile TypeScript to JavaScript before starting the development server.
In the following screenshot, the first five steps involve compiling the code to JavaScript and copying some files to the build
folder to start the HTTP server.
Even though the process seems logical, it has a lot of rough edges that will bite you once in a while. Many users created GitHub discussion threads lately expressing:
- I have updated my migrations code and it is not picked up by the
node ace migration:run
command - I am getting the error
node ace make:migration
is not a command - and so on
All this confusion is a result of a stale build
folder and you have to make sure that one process is always running to keep the build output up to date.
Let's do something better
I have been banging my head lately to find some alternative which feels more natural and intuitive over this additional build step, and voila there is ts-node .
Ts-Node is a library to run typescript code directly without transpiling it first. But, I decided to not use it and instead write my version of it for the following reasons.
- I wanted to avoid Type checking completely. No one looks at the terminal for checking the typescript errors. We all rely on our editors to report the errors and hence there is no need to slow down the compilation process.
- Use aggressive caching to make subsequent runs faster. TS node doesn't support caching as of today. There are some open issues to add it, but since they do a lot more than
@adonisjs/require-ts
they have to handle all other use cases as well. - Creating API for the watchers to invalidate the cache for the changed/updated files.
How in-memory compilation works?
Now that you are aware of the reasons for not using the ts-node, let's expand upon how in-memory compilation works and also the entire cache mechanism built to make subsequent runs faster.
- Node.js has a thing called require extensions
. Using this you can tell Node to call a function anytime a file with the given extension is required. Babel, ts-node, and many other libraries use this to hook into the
require
lifecycle of Node.js - TypeScript compiler API has a transpileModule method. You can pass the typescript source code as a string to this method and it will return you the compiled JavaScript.
So, if you combine the above two you can compile the typescript code on the fly. However, typescript does take some time to compile the code and this can make restarts slow. Let's visualize a standard development workflow.
- You run the
node ace serve --watch
command. - AdonisJS hooks into the require lifecycle and take control of compiling the
typescript
files. - As your application is getting ready all the source code imported using the
import
statement is getting compiled on the fly. - Next, you make some modifications to a given file.
- The file watcher is notified and it restarts the HTTP server and hence all the previously compiled code is destroyed since it was in memory.
- You have changed just one file and now everything needs to be re-compiled again. BUMMER!
Using cache
On-disk caching is the only solution to avoid re-compiling the entire project after a single file change. The following are the steps we perform for caching.
- Begin by generating the md5 hash of the file contents and use the hash as the filename to save the compiled contents on the disk.
- Next time, if the hash is still the same we read the compiled source from the disk instead of using the typescript compiler API. To our surprise, generating the hash + reading file from the disk is faster than re-compiling the code in many cases.
- If the hash is different we just ignore the cache and use the typescript compiler API.
The term ignore the cache is important here. Since we don't remove the old cache there is going to be a time when the cache will end taking a lot of disk space.
To counter that, we expose an API
from the @adonisjs/require-ts
module that the file watchers can use to clear the entire cache or remove a single file from it.
Result
Finally, we end up with a development workflow that can run TypeScript source code directly and uses cache for faster restarts.
Validating environment variables
Environment variables play a very important role in our applications. Moreover, environment variables are not in control of our source code and we heavily rely on the outside factors to provide the correct values. For example:
- Using a plain text file
.env
during development. - Using the control panel of cloud services like Heroku and Cleavr.
- Even during the CI/CD process, the environment variables are injected using the control panel.
We believe that validating the environment variable early in the lifecycle of running the application is a better approach instead of running an unstable system with in-correct or missing values.
To get started, create an env.ts
file in the root of your application and paste the following code snippet inside it.
import Env from '@ioc:Adonis/Core/Env'
export default Env.rules({
HOST: Env.schema.string({ format: 'host' }),
PORT: Env.schema.number(),
APP_KEY: Env.schema.string(),
})
With the above file in place, AdonisJS will automatically load this file to perform the validations.
Getting intellisense
Since we are already performing the runtime validations, wouldn't it be great if we can also get IntelliSense for the validated environment variables? Well, we can:
Begin by creating a new file contracts/env.ts
and paste the following code snippet inside it.
declare module '@ioc:Adonis/Core/Env' {
type CustomTypes = typeof import('../env').default
interface EnvTypes extends CustomTypes {}
}
Now, you will get proper IntelliSense when using the Env
module.
- The
Env.rules
method extracts the types from the defined keys and the validation rules. - Inside the
contracts/env.ts
, we make use of the declaration merging and extend theEnvTypes
interface to use the types exported in theenv.ts
file.
Introducing AdonisJS REPL
REPL stands read–eval–print loop, a way to quickly execute single-line inputs and return the result. Node.js also has its REPL and to give it try, you can open up your terminal, type node
, and press enter.
Similar to the Node.js REPL, AdonisJS now also has its REPL with first-class primitives to let you interact with your application. To begin, install the @adonisjs/repl package from the registry.
npm i @adonisjs/repl
yarn add @adonisjs/repl
Next, run the following command to set up the package.
node ace invoke @adonisjs/repl
That's all! Now you can run the node ace repl
command to start the REPL session.
Using Japa?
Projects setup to use Japa test runner
also have to update the japaFile.ts
file.
-
Update the files glob to remove the
build
prefix and use.ts
as the file extension.japaFile.tsconfigure({files: ['test/**/*.spec.ts'],// ...}) -
Update the path of
ADONIS_ACE_CWD
to use the following value.process.env.ADONIS_ACE_CWD = join(__dirname) -
Finally, update the script responsible for executing tests to run the
japaFile.ts
file directly as follows:node -r @adonisjs/assembler/build/register japaFile.ts