Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://docs.powertools.aws.dev/lambda/typescript/latest/features/jmespath/ below:

JMESPath Functions - Powertools for AWS Lambda (TypeScript)

Built-in JMESPath functions to easily deserialize common encoded JSON payloads in Lambda functions.

• Deserialize JSON from JSON strings, base64, and compressed data
• Use JMESPath to extract and combine data recursively
• Provides commonly used JMESPath expressions with popular event sources

Install the utility in your project:

npm install @aws-lambda-powertools/jmespath

You might have events that contain encoded JSON payloads as string, base64, or even in compressed format. It is a common use case to decode and extract them partially or fully as part of your Lambda function invocation.

Powertools for AWS Lambda (TypeScript) also have utilities like idempotency where you might need to extract a portion of your data before using them.

Envelope is the terminology we use for the JMESPath expression to extract your JSON object from your data input. We might use those two terms interchangeably.

You can use the extractDataFromEnvelope function with any JMESPath expression.

Another common use case is to fetch deeply nested data, filter, flatten, and more.

extractDataFromBuiltinEnvelope.tsextractDataFromEnvelope.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

import { extractDataFromEnvelope } from '@aws-lambda-powertools/jmespath/envelopes';

type MyEvent = {
  body: string; // "{\"customerId\":\"dd4649e6-2484-4993-acb8-0f9123103394\"}"
  deeplyNested: Array<{ someData: number[] }>;
};

type MessageBody = {
  customerId: string;
};

export const handler = async (event: MyEvent): Promise<unknown> => {
  const payload = extractDataFromEnvelope<MessageBody>(
    event,
    'powertools_json(body)'
  );
  const { customerId } = payload; // now deserialized

  // also works for fetching and flattening deeply nested data
  const someData = extractDataFromEnvelope<number[]>(
    event,
    'deeplyNested[*].someData[]'
  );

  return {
    customerId,
    message: 'success',
    context: someData,
    statusCode: 200,
  };
};

{
  "body": "{\"customerId\":\"dd4649e6-2484-4993-acb8-0f9123103394\"}",
  "deeplyNested": [
    {
      "someData": [1, 2, 3]
    }
  ]
}

We provide built-in envelopes for popular AWS Lambda event sources to easily decode and/or deserialize JSON objects.

extractDataFromBuiltinEnvelope.tsextractDataFromBuiltinEnvelope.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

import {
  extractDataFromEnvelope,
  SQS,
} from '@aws-lambda-powertools/jmespath/envelopes';
import { Logger } from '@aws-lambda-powertools/logger';
import type { SQSEvent } from 'aws-lambda';

const logger = new Logger();

type MessageBody = {
  customerId: string;
};

export const handler = async (event: SQSEvent): Promise<void> => {
  const records = extractDataFromEnvelope<Array<MessageBody>>(event, SQS);
  for (const record of records) {
    // records is now a list containing the deserialized body of each message
    const { customerId } = record;
    logger.appendKeys({ customerId });
  }
};

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

{
  "Records": [
    {
      "messageId": "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
      "receiptHandle": "MessageReceiptHandle",
      "body": "{\"customerId\":\"dd4649e6-2484-4993-acb8-0f9123103394\",\"booking\":{\"id\":\"5b2c4803-330b-42b7-811a-c68689425de1\",\"reference\":\"ySz7oA\",\"outboundFlightId\":\"20c0d2f2-56a3-4068-bf20-ff7703db552d\"},\"payment\":{\"receipt\":\"https://pay.stripe.com/receipts/acct_1Dvn7pF4aIiftV70/ch_3JTC14F4aIiftV700iFq2CHB/rcpt_K7QsrFln9FgFnzUuBIiNdkkRYGxUL0X\",\"amount\":100}}",
      "attributes": {
        "ApproximateReceiveCount": "1",
        "SentTimestamp": "1523232000000",
        "SenderId": "123456789012",
        "ApproximateFirstReceiveTimestamp": "1523232000001"
      },
      "messageAttributes": {},
      "md5OfBody": "7b270e59b47ff90a553787216d55d91d",
      "eventSource": "aws:sqs",
      "eventSourceARN": "arn:aws:sqs:us-east-1:123456789012:MyQueue",
      "awsRegion": "us-east-1"
    }
  ]
}

These are all built-in envelopes you can use along with their expression as a reference:

If you don't require SNS metadata, enable raw message delivery. It will reduce multiple payload layers and size, when using SNS in combination with other services (e.g., SQS, S3, etc).

You can use our built-in JMESPath functions within your envelope expression. They handle deserialization for common data formats found in AWS Lambda event sources such as JSON strings, base64, and uncompress gzip data.

Use powertools_json function to decode any JSON string anywhere a JMESPath expression is allowed.

Idempotency scenario

This sample will deserialize the JSON string within the body key before Idempotency processes it.

powertoolsJsonIdempotencyJmespath.tspowertoolsJsonIdempotencyJmespath.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

import { randomUUID } from 'node:crypto';
import {
  IdempotencyConfig,
  makeIdempotent,
} from '@aws-lambda-powertools/idempotency';
import { DynamoDBPersistenceLayer } from '@aws-lambda-powertools/idempotency/dynamodb';
import type { APIGatewayEvent } from 'aws-lambda';

const persistenceStore = new DynamoDBPersistenceLayer({
  tableName: 'IdempotencyTable',
});

export const handler = makeIdempotent(
  async (event: APIGatewayEvent) => {
    const body = JSON.parse(event.body || '{}');
    const { user, productId } = body;

    const result = await createSubscriptionPayment(user, productId);

    return {
      statusCode: 200,
      body: JSON.stringify({
        paymentId: result.id,
        message: 'success',
      }),
    };
  },
  {
    persistenceStore,
    config: new IdempotencyConfig({
      eventKeyJmesPath: 'powertools_json(body)',
    }),
  }
);

const createSubscriptionPayment = async (
  user: string,
  productId: string
): Promise<{ id: string; message: string }> => {
  const payload = { user, productId };
  const response = await fetch('https://httpbin.org/anything', {
    method: 'POST',
    body: JSON.stringify(payload),
  });

  if (!response.ok) {
    throw new Error('Failed to create subscription payment');
  }

  return { id: randomUUID(), message: 'paid' };
};

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

{
  "version": "2.0",
  "routeKey": "ANY /createpayment",
  "rawPath": "/createpayment",
  "rawQueryString": "",
  "headers": {
    "Header1": "value1",
    "Header2": "value2"
  },
  "requestContext": {
    "accountId": "123456789012",
    "apiId": "api-id",
    "domainName": "id.execute-api.us-east-1.amazonaws.com",
    "domainPrefix": "id",
    "http": {
      "method": "POST",
      "path": "/createpayment",
      "protocol": "HTTP/1.1",
      "sourceIp": "ip",
      "userAgent": "agent"
    },
    "requestId": "id",
    "routeKey": "ANY /createpayment",
    "stage": "$default",
    "time": "10/Feb/2021:13:40:43 +0000",
    "timeEpoch": 1612964443723
  },
  "body": "{\"user\":\"xyz\",\"product_id\":\"123456789\"}",
  "isBase64Encoded": false
}

Use powertools_base64 function to decode any base64 data.

This sample will decode the base64 value within the data key, and deserialize the JSON string before processing.

powertoolsBase64Jmespath.tspowertoolsBase64JmespathPayload.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import { extractDataFromEnvelope } from '@aws-lambda-powertools/jmespath/envelopes';
import { Logger } from '@aws-lambda-powertools/logger';

const logger = new Logger();

export const handler = async (event: { payload: string }): Promise<void> => {
  const data = extractDataFromEnvelope<string>(
    event,
    'powertools_json(powertools_base64(payload))'
  );

  logger.info('Decoded payload', { data }); // (1)!
};

1. The data variable contains the decoded object that looks like this:

   {
       user_id: 123,
       product_id: 1,
       quantity: 2,
       price: 10.4,
       currency: 'USD',
   }
   

{
  "payload": "eyJ1c2VyX2lkIjogMTIzLCAicHJvZHVjdF9pZCI6IDEsICJxdWFudGl0eSI6IDIsICJwcmljZSI6IDEwLjQwLCAiY3VycmVuY3kiOiAiVVNEIn0="
}

Use powertools_base64_gzip function to decompress and decode base64 data.

This sample will decompress and decode base64 data from Cloudwatch Logs, then use JMESPath pipeline expression to pass the result for decoding its JSON string.

powertoolsBase64GzipJmespath.tspowertoolsBase64GzipJmespathPayload.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import { extractDataFromEnvelope } from '@aws-lambda-powertools/jmespath/envelopes';
import { Logger } from '@aws-lambda-powertools/logger';

const logger = new Logger();

export const handler = async (event: { payload: string }): Promise<void> => {
  const logGroup = extractDataFromEnvelope<string>(
    event, // (1)!
    'powertools_base64_gzip(payload) | powertools_json(@).logGroup'
  );

  logger.info('Log group name', { logGroup }); // (2)!
};

1. The payload key contains a JSON object that once decompressed and decoded looks like this:

    1
    2
    3
    4
    5
    6
    7
    8
    9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25

   {
       "owner": "123456789012",
       "logGroup": "/aws/lambda/powertools-example",
       "logStream": "2020/09/02/[$LATEST]d3a8dcaffc7f4de2b8db132e3e106660",
       "subscriptionFilters": ["Destination"],
       "messageType": "DATA_MESSAGE",
       "logEvents": [
           {
               "id": "eventId1",
               "message": {
                   "username": "lessa",
                   "message": "hello world"
               },
               "timestamp": 1440442987000
           },
           {
               "id": "eventId2",
               "message": {
                   "username": "dummy",
                   "message": "hello world"
               },
               "timestamp": 1440442987001
           }
       ]
   }
   

2. The logGroup variable contains the string "/aws/lambda/powertools-example".

{
  "payload": "H4sIACZAXl8C/52PzUrEMBhFX2UILpX8tPbHXWHqIOiq3Q1F0ubrWEiakqTWofTdTYYB0YWL2d5zvnuTFellBIOedoiyKH5M0iwnlKH7HZL6dDB6ngLDfLFYctUKjie9gHFaS/sAX1xNEq525QxwFXRGGMEkx4Th491rUZdV3YiIZ6Ljfd+lfSyAtZloacQgAkqSJCGhxM6t7cwwuUGPz4N0YKyvO6I9WDeMPMSo8Z4Ca/kJ6vMEYW5f1MX7W1lVxaG8vqX8hNFdjlc0iCBBSF4ERT/3Pl7RbMGMXF2KZMh/C+gDpNS7RRsp0OaRGzx0/t8e0jgmcczyLCWEePhni/23JWalzjdu0a3ZvgEaNLXeugEAAA=="
}

This should only be used for advanced use cases where you have special formats not covered by the built-in functions.

For special binary formats that you want to decode before processing, you can bring your own JMESPath function by extending the PowertoolsFunctions class.

Here is an example of how to decompress messages compressed using the Brotli compression algorithm:

PowertoolsCustomFunction.tspowertoolsCustomFunction.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

 import { brotliDecompressSync } from 'node:zlib';
 import { fromBase64 } from '@aws-lambda-powertools/commons/utils/base64';
 import { extractDataFromEnvelope } from '@aws-lambda-powertools/jmespath/envelopes';
 import { PowertoolsFunctions } from '@aws-lambda-powertools/jmespath/functions';
 import { Logger } from '@aws-lambda-powertools/logger';

 const logger = new Logger();

 class CustomFunctions extends PowertoolsFunctions {
   @PowertoolsFunctions.signature({ // (1)!
     argumentsSpecs: [['string']],
     variadic: false,
   })
   public funcDecodeBrotliCompression(value: string): string { // (2)!
     const encoded = fromBase64(value, 'base64');
     const uncompressed = brotliDecompressSync(encoded);

     return uncompressed.toString();
   }
 }

 export const handler = async (event: { payload: string }): Promise<void> => {
   const message = extractDataFromEnvelope<string>(
     event,
     'Records[*].decode_brotli_compression(notification) | [*].powertools_json(@).message',
     { customFunctions: new CustomFunctions() }
   );

   logger.info('Decoded message', { message });
 };

1. The function signature can be enforced at runtime by using the @Functions.signature decorator.
2. The name of the function must start with the func prefix.

{
  "Records": [
    {
      "application": "app",
      "datetime": "2022-01-01T00:00:00.000Z",
      "notification": "GyYA+AXhZKk/K5DkanoQSTYpSKMwwxXh8DRWVo9A1hLqAQ=="
    }
  ]
}

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4