Saying AWS IoT FleetWise object storage in Amazon S3


At present, we’re excited to announce that AWS IoT FleetWise now helps object storage in Amazon Easy Storage Service (Amazon S3). This new function makes it straightforward and cost-effective for automotive clients to create and handle information pipelines from their automobiles. A buyer can now choose the place automobile information is endured within the cloud relying on their particular use case for that automobile information. AWS IoT FleetWise permits clients to gather, remodel, and switch automobile information to the cloud and use that information to enhance automobile high quality, electrification, and autonomy.

Automotive firms are looking for extra environment friendly methods to simplify information assortment from the automobiles. Amazon S3 help for AWS IoT FleetWise helps optimize the price of information storage and in addition present extra mechanisms to make use of automobile information inside a performant information lake, centralized information storage, information processing pipelines, visualization dashboards, and different enhancements to downstream information providers. Amazon S3 gives highly-performant and sturdy information administration capabilities which helps with unlocking new income alternatives from fleets, constructing machine studying datasets, and creating predictive upkeep fashions to detect and resolve issues in near-real time. Automotive firms can use these new capabilities to realize insights on issues like driving behaviors, infotainment interactions, and long-term upkeep wants for electrical automobile (EV) fleets.

Sending information from the automobile to Amazon S3 will allow automotive information engineers and information scientists to entry saved automobile information within the format required to investigate and enrich the info. Amazon S3 object storage for AWS IoT FleetWise helps two business commonplace information codecs for giant information implementations: Apache Parquet and JavaScript Object Notation (JSON). JSON is an ordinary human readable text-based format for representing structured information utilizing JavaScript object syntax. Prospects can use this format when they should preserve relational information within the payload, although there’s slight storage and compute overhead to implementing this format. Most information engineers will use Apache Parquet  format for vehicular telemetry information as it’s an open supply, versatile, and scalable format providing environment friendly information storage and retrieval. The format is appropriate for information compression and encoding schemes in quite a lot of frequent programming languages.

At launch in September 2022, AWS IoT FleetWise supplied Amazon Timestream as a knowledge persistence mechanism, which is primarily constructed to display and analyze how information modifications over time, offering the flexibility to determine tendencies and patterns in near-real time (time-series information). Amazon Timestream gives a close to real-time use instances which can provide, for instance, fleet operators a holistic view of their telemetry information by way of a marketing campaign deployed by AWS IoT FleetWise. Now, with Amazon S3, clients can unlock On-line Analytical Processing (OLAP) capabilities by way of batch information evaluation with multi-dimensional information factors. This functionality—switching from streaming information analytics to a extra batch information processing system—permits for the identification and remediation of issues in near-real time. It additionally helps to constantly enhance utilizing historic information from throughout fleets of automobiles, creating differentiation for the operator implementing predictive upkeep of their fleet.

Knowledge engineers can now implement software units utilizing their frequent information processes to extract, remodel, and cargo the info into an automotive information lake from a number of completely different sources of information, offering a centralized OLAP retailer for information scientists. This flexibility permits information engineers to carry automobile information instantly into different AWS providers like Amazon Athena and AWS Glue, which give ample alternatives to reinforce and enrich the telemetry information. Utilizing providers like Amazon Athena and AWS Glue additionally permits for formatting this information to be used inside machine studying fashions. For instance, clients can constantly enhance their predictive upkeep fashions, vary estimates, or energy-based routing for EV batteries based mostly on information saved in Amazon S3 from a battery monitoring system (BMS).

Hyundai Motor Group is innovating new options

Hyundai Motor Group (HMG) is a world automobile producer that provides shoppers a technology-rich lineup of vehicles, sport utility automobiles, and electrified automobiles. “At Hyundai, we’re targeted on utilizing the info we accumulate from automobiles to drive revolutionary infotainment options for our clients,” stated Youngwoo Park, vice chairman and head of the Infotainment Improvement Group at HMG. “With extra information administration choices accessible for AWS IoT FleetWise and the supply of Amazon S3, we’ll now be capable to course of batch information along with streaming information, giving us extra methods to know and unlock the total worth of car information.”

Nationwide Devices enhances EV battery monitoring

An AWS Companion, Nationwide Devices, will use AWS IoT FleetWise with Amazon S3 to reinforce their OptimalPlus answer on AWS by constructing a steady enchancment information pipeline for his or her inference fashions on electrical automobile batteries. The answer permits NI’s information scientists to make the most of the battery information which is aggregated from the BMS in-vehicle with AWS IoT FleetWise to constantly enhance electrical automobile predictive upkeep fashions. These fashions can then be deployed to the automobile, permitting automakers to dynamically modify settings within the BMS to increase the remaining helpful lifetime of the battery. “Constructing a knowledge ingestion and information pipeline workflow for battery monitoring methods with AWS IoT FleetWise has given us near-real time entry to electrical automobile information. Now, with AWS IoT FleetWise help for Amazon S3, our information engineers will get the batched information in an extensible, versatile, and cost-efficient method previous to bringing that information into our inference fashions,” stated Thomas Benjamin, CTO and Head of Platform and Analytics R&D at Nationwide Devices.

Resolution Overview

Let’s take a predictive upkeep use case to stroll you thru the method of making and deploying an AWS IoT FleetWise marketing campaign that shops information in Amazon S3. Think about you’re a information scientist at a fleet operator with 1000’s of supply vans. You’ve got the purpose to decrease the prices of brake system repairs and maximize automobile uptime. To do that, you may have constructed a machine studying mannequin that predicts when the pads will put on out. The mannequin requires you to assemble a complete dataset from varied sources corresponding to automobile upkeep historical past and the kind of brake pads used. Nevertheless, you might be lacking historic information on hard-braking occasions that may enhance the prediction accuracy. With information storage help for Amazon S3, AWS IoT FleetWise can now assist you to resolve this downside. You’ll create a condition-based marketing campaign that instructs your Edge Agent for AWS IoT FleetWise to seize 4 seconds of information earlier than and 1 second after a hard-braking occasion and retailer it in your S3 bucket in compressed Parquet format.


Earlier than you get began, you will want:

  • An AWS account with console and programmatic entry in supported Areas.
  • Permission to create and entry AWS IoT FleetWise and Amazon S3 assets.
  • To finish the AWS IoT FleetWise fast begin demo to set-up the simulation and all conditions earlier than making a marketing campaign.


Step 1: Create and deploy a condition-based marketing campaign that uploads a set of broadcast CAN alerts to your goal S3 bucket

1.1. Navigate to AWS IoT FleetWise console, choose Campaigns (left panel), select Create.

1.2. Configure marketing campaign: Set the marketing campaign identify to fwdemo-eventbased-s3-parquet-gzip

1.3. Select the Outline information assortment scheme and the Situation-based choice along with your particular person Marketing campaign length. Enter $variable.`Automobile.ABS.DemoBrakePedalPressure` > 7000 in Logical Expression and go away the optionally available settings as-is.

Define data scheme

Within the Superior scheme choices part, set the Submit set off assortment length as 1000 milliseconds.

Advanced scheme options

Within the Alerts to gather part, specify the alerts “Automobile.ECM.DemoEngineTorque” and “Automobile.ABS.DemoBrakePedalPressure.” The simulator generates a CAN message that carries the brake pedal place sign at 50 millisecond frequency. Max pattern depend of 100 and Min sampling interval of 0, instructs your Edge Agent to gather 5000 milliseconds of information that features 4000 milliseconds value of pre-event information and 1000 milliseconds value of post-event information.

Signals to collect

1.4. Outline storage vacation spot: Choose Amazon S3.

Define storage destination

Guarantee the next bucket coverage is utilized to your S3 bucket (change the $bucketName with the identify of your S3 bucket).

  "Model": "2012-10-17",
  "Assertion": [
      "Effect": "Allow",
      "Principal": {
        "Service": [
      "Motion": [
      "Useful resource": "arn:aws:s3:::$bucketName"
      "Impact": "Enable",
      "Principal": {
        "Service": [
      "Motion": [
      "Useful resource": "arn:aws:s3:::$bucketName/*"

Choose Parquet because the output format with the default GZIP compression.

Parquet output

1.5. Add automobiles: The simulated automobile from step 1 will present up right here as fwdemo.

Add Vehicles

1.6. Assessment and create: Assessment the settings, click on Create. After the standing change, click on Deploy to get your marketing campaign to your Edge Agent working in your simulated automobile.

Get campaign

1.7. Test information: Navigate to your S3 bucket to see your compressed Parquet information touchdown on the bucket each 12 to fifteen minutes as AWS IoT FleetWise completes its batch write-process.

Check S3 data

Step 2: Examine the collected information

For enterprise insights, you may question your compressed Parquet information with AWS Glue and Amazon Athena, and use Amazon QuickSight to visualise patterns within the hard-braking occasions.

Query Parquet data

Our automobile has generated a complete of seven.71K occasions throughout 11 hours of simulation. Right here, now we have created a easy visible that signifies a hard-braking situation by way of an abrupt spike in brake pedal stress and a drop in engine torque. Over time, this information will present priceless historic information you may mix with different datasets corresponding to automobile upkeep historical past, brake pad sort, and automobile weight to enhance the accuracy of your machine studying mannequin.

Visualize events

Now, that you’ve verified your marketing campaign, you may increase it to 1000’s of your vans to gather extra information and optimize your schedule for brake upkeep. To additional improve the accuracy of your mannequin, you may accumulate extra alerts corresponding to pace, harsh acceleration, or abrupt turns.

Cleansing up

Make sure you delete the next assets out of your AWS account to keep away from unintended expenses.

  1. Automobile Simulation assets within the CloudFormation console (fwdemo stack).
  2. Amazon Timestream assets with identify prefixes fwdemo within the Timestream console.
  3. Amazon S3 bucket.
  4. Marketing campaign within the AWS IoT FleetWise console.


On this submit, we showcased how AWS IoT FleetWise expands the scope of data-driven use instances for our automotive clients with the newly launched functionality of sending automobile information to Amazon S3. Along with the close to real-time monitoring and evaluation supplied by Amazon Timestream, the combination with Amazon S3 permits highly effective OLAP use instances corresponding to huge information evaluation and machine studying mannequin coaching. We then used a pattern predictive upkeep use case to stroll you thru the method of making a condition-based marketing campaign that collects hard-braking occasion information and sends it to Amazon S3.

To be taught extra, go to the AWS IoT FleetWise web site or login to the console to get began. We sit up for your suggestions and questions.

Andrew Givens

Andrew Givens

Andrew is a IoT Specialist at Amazon Net Companies. Primarily based in Atlanta, he helps world automotive clients construct their related automobile options on AWS IoT. With deep expertise within the automotive business, he has a specific curiosity in extensible, scalable, automobile communication platforms on AWS.

Jay Chung

Jay Chung

Jay is an IoT Architect working within the IoT International Specialty Apply in AWS Skilled Companies. Jay loves participating with clients to construct IoT options that assist clients resolve their enterprise challenges. Previous to becoming a member of AWS, Jay spent over a decade serving a number of roles within the automotive take a look at software business together with software program improvement and product administration.

Leave a Reply

Your email address will not be published. Required fields are marked *