til.duyet.net
blog
Search…
til.duyet.net
Today I Learned
Data Engineer
AWS
Tools
Shell
EMR
Kubernetes
Helm Charts
Apache Airflow
Docker
Database
Google BigQuery
Apache Hive
AWS Redshift / Postgres
Presto
Python
date_range_generator
get_all_s3_keys
Pipenv
YAML config file with environment variables
Unix
Git - Pretty git branch graphs
Checking files in Docker build context
Bash get the directory of the current script
Vim
Find and replace
Articles
Miscellaneous
[Fig] Single machine and distributed system structure
Deploying Machine Learning Models at Scale
Bypass a Chrome certificate/HSTS error
Go
Functions and Methods in Go?
Convert JSON to Go struct
Frontend / React
Flexbox
Create hook to inject JS script
Javascript / Typescript
Intl.ListFormat
Powered By
GitBook
Tools
Dremio -
The Data Lake Engine
-
https://www.dremio.com
Apache Airflow
-
programmatically author, schedule and monitor workflows
-
https://airflow.apache.org
Perfect -
The New Standard in Dataflow Automation
-
https://www.prefect.io
Flyte -
Develop, execute, and monitor distributed workflows reliably at scale
-
https://github.com/lyft/flyte
Amundsen -
Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting with data
- https://github.com/lyft/amundsen
Previous
Make an NVMe volume available for use on AWS EC2
Next - Data Engineer
Shell
Last modified
1yr ago
Copy link