Data Analysis and Visualization
Data Analysis and Visualization
Embedded Files
Enhance and simplify current TrailCam visualization (custom JavaScript/Node/D3) by using BigQuery and Looker Studio
Ingest json data files (python, json)
Transform into BigQuery compatible schema (python, pandas)
Write data to BigQuery (python, google-cloud-bigquery module)
Connect Looker Studio to BigQuery via GCP-native BigQuery integration
Analyze and Visualize the data in Looker
You can
Look at visualization on the data using Looker Studio.
Check out the BigQuery integration code at github.com/boaworm/TrailcamScripts
End result: Comparing the three ways to analyze and visualize:
Pure JavaScript/Vue
+ Much faster rendering
+ More customization options (like graphs with empty bars for hours without data : 0-23)
- longer time to code and maintain.
Looker Studio ingesting JSON via public Connector.
+ Easy to generate graphs.
- Relatively slow to refresh (seconds to load).
- Inability to support certain custom features like empty bars.
- Lacking support for various datatypes (date, sequence).
Looker Studio connected to BigQuery.
+ Easy to generate and maintain graphs.
+ Faster to refresh than direct JSON ingestion.
+ No need to use 3rd party connector (built-in Looker Studio <> BQ support)
+ Using a custom schema and python+pandas for data transformation, I could correctly ingest as DATE, REPEAT etc.
Page updated
Report abuse