Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://www.gatsbyjs.com/plugins/gatsby-source-filesystem/ below:

gatsby-source-filesystem | Gatsby

A Gatsby plugin for sourcing data into your Gatsby application from your local filesystem.

The plugin creates File nodes from files. The various transformer plugins can transform File nodes into other types of data e.g. gatsby-transformer-json transforms JSON files into JSON nodes and gatsby-transformer-remark transforms markdown files into MarkdownRemark nodes.

npm install gatsby-source-filesystem

You can have multiple instances of this plugin in your gatsby-config to read files from different locations on your filesystem. Be sure to give each instance a unique name.

gatsby-config.js

module.exports = {
  plugins: [
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        
        name: `pages`,
        
        path: `${__dirname}/src/pages/`,
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: `data`,
        path: `${__dirname}/src/data/`,
        
        ignore: [`**/\.*`],
        
        fastHash: true,
      },
    },
  ],
}

In the above example every file under src/pages and src/data will be made available as a File node inside GraphQL. You don’t need to set up another instance of gatsby-source-filesystem for e.g. src/data/images (since those files are already sourced). However, if you want to be able to filter your files you can set up a new instance and later use the sourceInstanceName.

Required

A unique name for the gatsby-source-filesytem instance. This name will also be a key on the File node called sourceInstanceName. You can use this e.g. for filtering.

Required

Path to the folder that should be sourced. Ideally an absolute path.

Optional

Array of file globs to ignore. They will be added to the following default list:

**/*.un~
**/.DS_Store
**/.gitignore
**/.npmignore
**/.babelrc
**/yarn.lock
**/node_modules
../**/dist/**

Optional

By default, gatsby-source-filesystem creates an MD5 hash of each file to determine if it has changed between sourcing. However, on sites with many large files this can lead to a significant slowdown. Thus you can enable the fastHash setting to use an alternative hashing mechanism.

fastHash uses the mtime and inode to fingerprint the files. On a modern OS this can be considered a robust solution to determine if a file has changed, however on older systems it can be unreliable. Therefore it’s not enabled by default.

• GATSBY_CONCURRENT_DOWNLOAD (default: 200). To prevent concurrent requests you can configure the concurrency of processRemoteNode.

If you have a spotty network or slow connection, you can adjust the retries and timeouts:

• GATSBY_STALL_RETRY_LIMIT (default: 3)
• GATSBY_STALL_TIMEOUT (default: 30000)
• GATSBY_CONNECTION_TIMEOUT (default: 30000)

You can query the File nodes as following:

{
  allFile {
    nodes {
      extension
      dir
      modifiedTime
    }
  }
}

Use GraphiQL to explore all available keys.

To filter by the name you specified in the gatsby-config, use sourceInstanceName:

{
  allFile(filter: { sourceInstanceName: { eq: "data" } }) {
    nodes {
      extension
      dir
      modifiedTime
    }
  }
}

gatsby-source-filesystem exports three helper functions:

• createFilePath
• createRemoteFileNode
• createFileNodeFromBuffer

When building pages from files, you often want to create a URL from a file’s path on the filesystem. For example, if you have a markdown file at src/content/2018-01-23-my-blog-post/index.md, you might want to turn that into a page on your site at example.com/blog/2018-01-23-my-blog-post/. createFilePath is a helper function to make this task easier.

createFilePath({
  
  
  node,
  
  
  getNode,
  
  
  
  basePath,
  
  
  trailingSlash,
})

gatsby-node.js

const { createFilePath } = require(`gatsby-source-filesystem`)

exports.onCreateNode = ({ node, getNode, actions }) => {
  const { createNodeField } = actions
  
  if (node.internal.type === "MarkdownRemark") {
    
    const relativeFilePath = createFilePath({
      node,
      getNode,
      basePath: "src/content",
    })

    
    createNodeField({
      node,
      name: "slug",
      value: `/blog${relativeFilePath}`,
    })
  }
}

When building source plugins for remote data sources (Headless CMSs, APIs, etc.), their data will often link to files stored remotely that are often convenient to download so you can work with them locally.

The createRemoteFileNode helper makes it easy to download remote files and add them to your site’s GraphQL schema.

While downloading the assets, special characters (regex: /:|\/|\*|\?|"|<|>|\||\\/g) in filenames are replaced with a hyphen ”-“. When special characters are found a file hash is added to keep files unique e.g a:file.jpg becomes a-file-73hd.jpg (as otherwise a:file.jpg and a*file.jpg would overwrite themselves).

createRemoteFileNode({
  
  url: `https://example.com/a-file.jpg`,
  
  parentNodeId,
  
  getCache,
  
  createNode,
  
  createNodeId,
  
  
  auth: { htaccess_user: `USER`, htaccess_pass: `PASSWORD` },
  
  
  httpHeaders: { Authorization: `Bearer someAccessToken` },
  
  
  ext: `.jpg`,
})

The following example is pulled from the Preprocessing External Images guide. Downloaded files are created as File nodes and then linked to the MarkdownRemark node, so it can be used with e.g. gatsby-plugin-image. The file node can then be queried using GraphQL.

gatsby-node.js

const { createRemoteFileNode } = require("gatsby-source-filesystem")

exports.onCreateNode = async ({
  node,
  actions: { createNode, createNodeField },
  createNodeId,
  getCache,
}) => {
  
  if (
    node.internal.type === "MarkdownRemark" &&
    node.frontmatter.featuredImgUrl !== null
  ) {
    const fileNode = await createRemoteFileNode({
      url: node.frontmatter.featuredImgUrl, 
      parentNodeId: node.id, 
      createNode, 
      createNodeId, 
      getCache,
    })

    
    if (fileNode) {
      createNodeField({ node, name: "localFile", value: fileNode.id })
    }
  }
}

The helper first tries to retrieve the file name and extension by parsing the url and the path provided (e.g. if the url is https://example.com/image.jpg, the extension will be inferred as .jpg and the name as image). If the url does not contain an extension, createRemoteFileNode use the file-type package to infer the file type. Finally, the name and the extension can be explicitly passed, like so:

createRemoteFileNode({
  
  url: `https://example.com/a-file-without-an-extension`,
  parentNodeId: node.id,
  getCache,
  createNode,
  createNodeId,
  
  ext: `.jpg`,
  name: `image`,
})

When working with data that isn’t already stored in a file, such as when querying binary/blob fields from a database, it’s helpful to cache that data to the filesystem in order to use it with other transformers that accept files as input.

The createFileNodeFromBuffer helper accepts a Buffer, caches its contents to disk, and creates a File node that points to it.

The name of the file can be passed to the createFileNodeFromBuffer helper. If no name is given, the content hash will be used to determine the name.

The following example is adapted from the source of gatsby-source-mysql:

gatsby-node.js

const createMySqlNodes = require(`./create-nodes`)

exports.sourceNodes = async ({ actions, createNodeId, getCache }, config) => {
  const { createNode } = actions
  const { conn, queries } = config
  const { db, results } = await query(conn, queries)

  try {
    queries
      .map((query, i) => ({ ...query, ___sql: results[i] }))
      .forEach(result =>
        createMySqlNodes(result, results, createNode, {
          createNode,
          createNodeId,
          getCache,
        })
      )
    db.end()
  } catch (e) {
    console.error(e)
    db.end()
  }
}


const { createFileNodeFromBuffer } = require(`gatsby-source-filesystem`)
const createNodeHelpers = require(`gatsby-node-helpers`).default

const { createNodeFactory } = createNodeHelpers({ typePrefix: `mysql` })

function attach(node, key, value, ctx) {
  if (Buffer.isBuffer(value)) {
    ctx.linkChildren.push(parentNodeId =>
      createFileNodeFromBuffer({
        buffer: value,
        getCache: ctx.getCache,
        createNode: ctx.createNode,
        createNodeId: ctx.createNodeId,
      })
    )
    value = `Buffer`
  }

  node[key] = value
}

function createMySqlNodes({ name, __sql, idField, keys }, results, ctx) {
  const MySqlNode = createNodeFactory(name)
  ctx.linkChildren = []

  return __sql.forEach(row => {
    if (!keys) keys = Object.keys(row)

    const node = { id: row[idField] }

    for (const key of keys) {
      attach(node, key, row[key], ctx)
    }

    node = ctx.createNode(node)

    for (const link of ctx.linkChildren) {
      link(node.id)
    }
  })
}

module.exports = createMySqlNodes

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4