A RetroSearch Logo

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Search Query:

Showing content from https://github.com/unifiedjs/unified-engine below:

unifiedjs/unified-engine: Engine to process multiple files with unified

unified engine to process multiple files, lettings users configure from the file system.

This package is the engine. It’s what you use underneath when you use remark-cli or a language server. Compared to unified, this deals with multiple files, often from the file system, and with configuration files and ignore files.

You typically use something that wraps this, such as:

You can use this to make such things.

This package is ESM only. In Node.js (version 16+), install with npm:

npm install unified-engine

The following example processes all files in the current folder with a markdown extension with remark, allows configuration from .remarkrc and package.json files, ignoring files from .remarkignore files, and more.

/**
 * @import {Callback} from 'unified-engine'
 */

import process from 'node:process'
import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    color: true,
    extensions: ['md', 'markdown', 'mkd', 'mkdn', 'mkdown'],
    files: ['.'],
    ignoreName: '.remarkignore',
    packageField: 'remarkConfig',
    pluginPrefix: 'remark',
    processor: remark,
    rcName: '.remarkrc'
  },
  done
)

/** @type {Callback} */
function done(error, code) {
  if (error) throw error
  process.exitCode = code
}

This package exports the identifiers Configuration and engine. There is no default export.

engine(options, callback?)

Process.

If a callback is given, nothing (undefined). Otherwise Promise<ContextWithCode>.

Internal class to load configuration files.

Exposed to build more complex integrations.

Completer (TypeScript type).

type Completer = (CompleterCallback | CompleterRegular) & {
    pluginId?: string | symbol | undefined
}

type CompleterCallback = (set: FileSet, next: CompleterCallbackNext) => undefined
type CompleterCallbackNext = (error?: Error | null | undefined) => undefined
type CompleterRegular = (set: FileSet) => Promise<undefined> | undefined

Callback called when done (TypeScript type).

Called with a fatal error if things went horribly wrong (probably due to incorrect configuration), or a status code and the processing context.

Nothing (undefined).

Resolved configuration from Configuration (TypeScript type).

Transform arbitrary configs to our format (TypeScript type).

Our config format (Preset).

Processing context with code (TypeScript type).

Processing context (TypeScript type).

A FileSet is created to process multiple files through unified processors (TypeScript type).

This set, containing all files, is exposed to plugins as the second parameter.

None.

Configuration (TypeScript type).

👉 Note: options.processor is required.

Sharable configuration, with support for specifiers (TypeScript type).

Specifiers should not be used in actual presets (because they can’t be used by regular unified), but they can be used in config files locally, as those are only for the engine.

They can contain plugins and settings.

import type {
  Plugin as UnifiedPlugin,
  PluginTuple as UnifiedPluginTuple,
  Preset as UnifiedPreset,
  Settings
} from 'unified'

type Preset = {
  plugins?: PluggableList | PluggableMap | undefined
  settings?: Settings | undefined
}

type Pluggable = Plugin | PluginTuple | UnifiedPreset
type PluggableList = Array<Pluggable>
type PluggableMap = Record<string, unknown>
type Plugin = UnifiedPlugin | string
type PluginTupleSupportingSpecifiers =
  | [plugin: string, ...parameters: Array<unknown>]
  | UnifiedPluginTuple

How to resolve (TypeScript type).

type ResolveFrom = 'cwd' | 'dir';

Transform arbitrary configs to our format (TypeScript type).

This is essentially the interface of vfile-reporter, with added support for unknown fields in options and async support.

Report (Promise<string> or string).

unified-engine accepts configuration through options and through configuration files (rc files).

One configuration file can be given through options.rcPath, this is loaded regardless of options.detectConfig and options.rcName.

Otherwise, configuration files are detected if options.detectConfig is turned on, depending on the following options:

The first file that is searched for in a folder is used as the configuration. If no file is found, the parent folder is searched, and so on.

The schema (type) of rc files is Preset.

An example rc file could look as follows:

{
  "plugins": [
    "remark-inline-links",
    "remark-lint-recommended"
  ],
  "settings": {
    "bullet": "*",
    "ruleRepetition": 3,
    "fences": true
  }
}

Another example, rc.js, could look as follows:

exports.plugins = [
  './script/natural-language.js',
  'remark-lint-recommended',
  'remark-license'
]

exports.settings = {bullet: '*'}

When using ESM (ECMAScript modules), rc.mjs could look as folows:

export default {
  plugins: [
    './script/natural-language.js',
    'remark-lint-recommended',
    'remark-license'
  ],
  settings: {bullet: '*'}
}

Another example, rc.yaml, could look as follows:

plugins:
  - 'rehype-document'
  - 'rehype-preset-minify'
settings:
  preferUnquoted: true
  quote: "'"
  quoteSmart: true
  verbose: true

unified-engine accepts patterns to ignore when searching for files to process through ignore files.

One ignore file can be given through options.ignorePath, this is loaded regardless of options.detectIgnore and options.ignoreName.

Otherwise, ignore files are detected if options.detectIgnore is turned on and options.ignoreName is given.

The first file named $ignoreName in the parent folder of a checked file is used. Or, if no file is found, the parent folder if searched, and so on.

In addition to explicit and implicit ignore files, other patterns can be given with options.ignorePatterns. The format of each pattern in ignorePatterns is the same as a line in an ignore file. Patterns and files are resolved based on the current working folder.

It is also possible to ignore files that do not have an associated detected configuration file by turning on options.ignoreUnconfigured.

Ignoring is used when searching for files in folders. If paths (including those expanded from globs) are passed in that are ignored, an error is thrown. These files can be silently ignored by turning on options.silentlyIgnore.

Normally, files are ignored based on the path of the found ignore file and the patterns inside it. Patterns passed with options.ignorePatterns are resolved based on the current working directory.

Patterns in an explicit ignore file passed in with options.ignorePath can be resolved from the current working directory instead, by setting options.ignorePathResolveFrom to 'cwd' instead of 'dir' (default).

If paths or globs to folders are given to the engine, they will be searched for matching files, but node_modules are normally not searched. Pass paths (or globs) to the node_modules you want to include in options.files to search them.

The format for ignore files is the same as .gitignore, so it’s possible to pass a .gitignore in as options.ignorePath.

node-ignore is used under the hood, see its documentation for more information.

An example ignore file could look as follows:

# Ignore files in `.github`.
.github/

# Bower.
bower_components/
# Duo dependencies.
components/

# Fixtures.
test/{input,tree}/

If we had an ignore file folder/.remarkignore, with the value: index.txt, and our file system looked as follows:

folder/.remarkignore
folder/index.txt
index.txt

Then folder/index.txt would be ignored but index.txt would not be.

Normally, unified plugins receive a single options argument upon attaching (an Object users can provide to configure the plugin).

If a plugin is attached by unified-engine, a second argument is given: FileSet.

unified-engine can be configured extensively by engine authors.

This example shows how you can use options.alwaysStringify when you don’t want the engine to write to the file system, but still want to get the compiled results. One example that does this is unified-engine-gulp.

import {remark} from 'remark'
import {engine} from 'unified-engine'
import {VFile} from 'vfile'

const file = new VFile({path: 'example.md', value: '_hi_'})

engine(
  {alwaysStringify: true, files: [file], processor: remark},
  function (error, code, context) {
    if (error) throw error
    console.log(context?.files.map((d) => String(d)))
  }
)

Yields:

example.md: no issues found

To support custom rc files, that have a different format than what the engine supports, pass as ConfigTransform.

This example processes readme.md and loads options from custom (from a package.json). configTransform is called with those options and transforms it to configuration unified-engine understands.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    configTransform,
    files: ['readme.md'],
    packageField: 'custom',
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

function configTransform(config) {
  return {settings: (config || {}).options}
}

Where package.json contains:

{
  "name": "foo",
  "private": true,
  "custom": {
    "options": {
      "bullet": "+"
    }
  }
}

This example processes readme.md. If package.json exists, that config is used, otherwise the configuration at defaultConfig is used.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    defaultConfig: {settings: {bullet: '+'}},
    files: ['readme.md'],
    packageField: 'remarkConfig',
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

Where package.json contains:

{
  "name": "foo",
  "private": true,
  "remarkConfig": {
    "settings": {
      "bullet": "-"
    }
  }
}

This example processes readme.md but does not allow configuration from .remarkrc or package.json files, as detectConfig is false.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    detectConfig: false,
    files: ['readme.md'],
    processor: remark(),
    packageField: 'remarkConfig',
    rcName: '.remarkrc'
  },
  function (error) {
    if (error) throw error
  }
)

This example processes files in the current working directory with an md extension but does not ignore file paths from the closest .remarkignore file, because detectIgnore is false.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    detectIgnore: false,
    extensions: ['md'],
    files: ['.'],
    ignoreName: '.remarkignore',
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

This example reformats all files with md, markdown, and mkd extensions in the current folder.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    extensions: ['md', 'mkd', 'markdown'],
    files: ['.'],
    output: true,
    processor: remark
  },
  function (error) {
    if (error) throw error
  }
)

This example shows that streamIn is named as filePath:

import {PassThrough} from 'node:stream'
import {remark} from 'remark'
import remarkPresetLintRecommended from 'remark-preset-lint-recommended'
import {engine} from 'unified-engine'

const streamIn = new PassThrough()

streamIn.write('doc')

setImmediate(function () {
  streamIn.end('ument')
})

engine(
  {
    filePath: '~/alpha/bravo/charlie.md',
    out: false,
    plugins: [remarkPresetLintRecommended],
    processor: remark(),
    streamIn
  },
  function (error) {
    if (error) throw error
  }
)

Yields:

~/alpha/bravo/charlie.md
  1:1  warning  Missing newline character at end of file  final-newline  remark-lint

⚠ 1 warning

This example processes LICENSE and all files with an md extension in doc.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    extensions: ['md'],
    files: ['LICENSE', 'doc/'],
    processor: remark
  },
  function (error) {
    if (error) throw error
  }
)

This example uses remark-lint to lint readme.md and exits with the given exit code. Normally, only errors turn the code to 1, but in frail mode lint warnings result in the same.

import process from 'node:process'
import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    files: ['readme.md'],
    frail: true,
    plugins: ['remark-preset-lint-recommended'],
    processor: remark()
  },
  function (error, code) {
    process.exitCode = error ? 1 : code
  }
)

This example processes files in the current working directory with an md extension, and is configured to ignore file paths from the closest .remarkignore file.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    extensions: ['md'],
    files: ['.'],
    ignoreName: '.remarkignore',
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

This example processes files in the current working directory with an md extension and ignores file paths specified in .gitignore.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    extensions: ['md'],
    files: ['.'],
    ignorePath: '.gitignore',
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)
options.ignorePathResolveFrom

This example processes files in the current working directory with an md extension and takes a reusable configuration file from a dependency.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    extensions: ['md'],
    files: ['.'],
    ignorePath: 'node_modules/my-config/my-ignore',
    ignorePathResolveFrom: 'cwd',
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

This example processes files in the current working directory with an md extension, except for readme.md:

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    extensions: ['md'],
    files: ['.'],
    ignorePatterns: ['readme.md'],
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)
options.ignoreUnconfigured

This example processes files in the current working directory with an md extension, but only if there is an explicit .remarkrc config file near (upwards) to them:

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    extensions: ['md'],
    files: ['.'],
    ignoreUnconfigured: true,
    processor: remark(),
    rcName: '.remarkrc'
  },
  function (error) {
    if (error) throw error
  }
)

This example shows a module which reads and parses doc.md, then remark-unlink transforms the syntax tree, the tree is formatted with unist-util-inspect, and finally written to stdout(4).

import {remark} from 'remark'
import remarkUnlink from 'remark-unlink'
import {engine} from 'unified-engine'

engine(
  {
    files: ['doc.md'],
    inspect: true,
    plugins: [remarkUnlink],
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

Where doc.md looks as follows:

[foo](https://example.com)

Yields:

root[1] (1:1-2:1, 0-27)
└─ paragraph[1] (1:1-1:27, 0-26)
   └─ text: "foo" (1:2-1:5, 1-4)

This example uses remark-lint to lint readme.md, writes the report, and ignores the serialized document.

import {remark} from 'remark'
import remarkPresetLintRecommended from 'remark-preset-lint-recommended'
import {engine} from 'unified-engine'

engine(
  {
    files: ['readme.md'],
    out: false,
    plugins: [remarkPresetLintRecommended],
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

This example writes all files in src/ with an md extension compiled to dest/.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    extensions: ['md'],
    files: ['src/'],
    output: 'dest/',
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

This example processes readme.md, and allows configuration from remarkConfig fields in package.json files.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    files: ['readme.md'],
    packageField: 'remarkConfig',
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

This example processes readme.md and loads the preset-lint-recommended plugin. Because pluginPrefix is given, this resolves to remark-preset-lint-recommended (from node_modules/) if available.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    files: ['readme.md'],
    pluginPrefix: 'remark',
    plugins: ['preset-lint-recommended'],
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

This example processes readme.md and loads the remark-preset-lint-recommended preset.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    files: ['readme.md'],
    plugins: ['remark-preset-lint-recommended'],
    processor: remark()
  },
  function (error) {
    if (error) throw error
  }
)

This example reformats stdin(4) using remark, writes the report to stderr(4), and formatted document to stdout(4).

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine({processor: remark}, function (error) {
  if (error) throw error
})

This example uses remark-lint to lint readme.md. Nothing is reported if the file processed successfully.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    files: ['readme.md'],
    plugins: ['remark-preset-lint-recommended'],
    processor: remark(),
    quiet: true
  },
  function (error) {
    if (error) throw error
  }
)

This example processes readme.md and allows configuration from .remarkrc, .remarkrc.json, .remarkrc.yml, .remarkrc.yaml, .remarkrc.js, .remarkrc.cjs, and .remarkrc.mjs files.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {files: ['readme.md'], processor: remark(), rcName: '.remarkrc'},
  function (error) {
    if (error) throw error
  }
)

This example processes readme.md and loads configuration from config.json.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {files: ['readme.md'], processor: remark(), rcPath: 'config.json'},
  function (error) {
    if (error) throw error
  }
)
options.reporter and options.reporterOptions

This example processes all HTML files in the current folder with rehype, configures the processor with .rehyperc files, and prints a report in JSON using vfile-reporter-json with reporter options.

import {rehype} from 'rehype'
import {engine} from 'unified-engine'

engine(
  {
    extensions: ['html'],
    files: ['.'],
    processor: rehype(),
    rcName: '.rehyperc',
    reporter: 'json',
    reporterOptions: {pretty: true}
  },
  function (error) {
    if (error) throw error
  }
)

This example processes readme.md and configures the compiler (remark-stringify) with bullet: '+'.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {files: ['readme.md'], processor: remark(), settings: {bullet: '+'}},
  function (error) {
    if (error) throw error
  }
)

This example uses remark-lint to lint readme.md but does not report any warnings or success messages, only fatal errors, if they occur.

import {remark} from 'remark'
import {engine} from 'unified-engine'

engine(
  {
    files: ['readme.md'],
    plugins: ['remark-preset-lint-recommended'],
    processor: remark(),
    silent: true
  },
  function (error) {
    if (error) throw error
  }
)

This example uses remark-lint to lint readme.md and writes the report to report.txt.

import fs from 'node:fs'
import {remark} from 'remark'
import remarkPresetLintRecommended from 'remark-preset-lint-recommended'
import {engine} from 'unified-engine'

engine(
  {
    files: ['readme.md'],
    out: false,
    plugins: [remarkPresetLintRecommended],
    processor: remark(),
    streamErr: fs.createWriteStream('report.txt')
  },
  function (error) {
    if (error) throw error
  }
)

This example uses remark-lint to lint an incoming stream.

import {PassThrough} from 'node:stream'
import {remark} from 'remark'
import remarkPresetLintRecommended from 'remark-preset-lint-recommended'
import {engine} from 'unified-engine'

const streamIn = new PassThrough()

streamIn.write('doc')

setImmediate(function () {
  streamIn.end('ument')
})

engine(
  {
    out: false,
    plugins: [remarkPresetLintRecommended],
    processor: remark(),
    streamIn
  },
  function (error) {
    if (error) throw error
  }
)

Yields:

<stdin>
  1:1  warning  Missing newline character at end of file  final-newline  remark-lint

⚠ 1 warning

This example reads readme.md and writes the serialized document to readme-two.md. This can also be achieved by passing output: 'readme-two.md' instead of streamOut.

import fs from 'node:fs'
import {remark} from 'remark'
import {engine} from 'unified-engine'

const streamOut = fs.createWriteStream('readme-two.md')

engine(
  {files: ['readme.md'], processor: remark(), streamOut},
  function (error) {
    if (error) throw error
  }
)

This example reads tree.json, then remark-unlink transforms the syntax tree, and the transformed tree is written to stdout(4).

import {remark} from 'remark'
import remarkUnlink from 'remark-unlink'
import {engine} from 'unified-engine'

engine(
  {
    files: ['tree.json'],
    plugins: [remarkUnlink],
    processor: remark(),
    tree: true
  },
  function (error) {
    if (error) throw error
  }
)

Where tree.json looks as follows:

{
  "type": "paragraph",
  "children": [{
    "type": "link",
    "url": "https://example.com",
    "children": [{
      "type": "text",
      "value": "foo"
    }]
  }]
}

Yields:

{
  "type": "paragraph",
  "children": [{
    "type": "text",
    "value": "foo"
  }]
}

This example reads tree.json, then remark-unlink transforms the syntax tree, the tree is serialized, and the resulting document is written to stdout(4).

import {remark} from 'remark'
import remarkUnlink from 'remark-unlink'
import {engine} from 'unified-engine'

engine(
  {
    files: ['tree.json'],
    plugins: [remarkUnlink],
    processor: remark(),
    treeIn: true
  },
  function (error) {
    if (error) throw error
  }
)

Where tree.json looks as follows:

{
  "type": "paragraph",
  "children": [{
    "type": "link",
    "url": "https://example.com",
    "children": [{
      "type": "text",
      "value": "foo"
    }]
  }]
}

Yields:

This example shows a module which reads and parses doc.md, then remark-unlink transforms the syntax tree, and the tree is written to stdout(4).

import {remark} from 'remark'
import remarkUnlink from 'remark-unlink'
import {engine} from 'unified-engine'

engine(
  {
    files: ['doc.md'],
    plugins: [remarkUnlink],
    processor: remark(),
    treeOut: true
  },
  function (error) {
    if (error) throw error
  }
)

Where doc.md looks as follows:

[foo](https://example.com)

Yields:

{
  "type": "paragraph",
  "children": [{
    "type": "text",
    "value": "foo"
  }]
}

This package is fully typed with TypeScript. It exports the additional types Completer, Callback, ConfigResult, ConfigTransform, ContextWithCode, Context, FileSet, Options, Preset, ResolveFrom, and VFileReporter.

Projects maintained by the unified collective are compatible with maintained versions of Node.js.

When we cut a new major release, we drop support for unmaintained versions of Node. This means we try to keep the current release line, unified-engine@^11, compatible with Node.js 16.

unified-engine loads and evaluates configuration files, plugins, and presets from the file system (often from node_modules/). That means code that is on your file system runs. Make sure you trust the workspace where you run unified-engine and be careful with packages from npm and changes made by contributors.

See contributing.md in unifiedjs/.github for ways to get started. See support.md for ways to get help.

This project has a code of conduct. By interacting with this repository, organization, or community you agree to abide by its terms.

MIT © Titus Wormer

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4