# til.duyet.net

## til.duyet.net

- [Today I Learned](https://til.duyet.net/master.md): I write to discover what I know. — Flannery O'Connor
- [AWS](https://til.duyet.net/data-engineering/aws.md)
- [Make an NVMe volume available for use on AWS EC2](https://til.duyet.net/data-engineering/aws/make-an-nvme-volume-available-for-use-on-aws-ec2.md)
- [AWS EMR](https://til.duyet.net/data-engineering/aws/emr.md): Manage EMR Cluster
- [S3 Dist CP](https://til.duyet.net/data-engineering/aws/emr/s3-dist-cp.md): Using S3DistCp, you can efficiently copy large amounts of data from Amazon S3/HDFS into S3/HDFS
- [Manage services](https://til.duyet.net/data-engineering/aws/emr/manage-services.md)
- [EMR - Tez](https://til.duyet.net/data-engineering/aws/emr/emr-tez.md): Upscale/Downscale of the emr cluster corrupts hdfs jars /apps/tez/tez.tar.gz
- [Issues](https://til.duyet.net/data-engineering/aws/emr/issues.md)
- [Tools](https://til.duyet.net/data-engineering/tools.md)
- [Shell](https://til.duyet.net/data-engineering/shell.md)
- [MacOS's Touch ID on Terminal](https://til.duyet.net/data-engineering/shell/macoss-touch-id-on-terminal.md)
- [Using \`sed\` to find and replace in file](https://til.duyet.net/data-engineering/shell/using-sed-to-find-and-replace-in-file.md): sed = Stream EDitor
- [Merging contents of multiple .csv files into single .csv file](https://til.duyet.net/data-engineering/shell/merging-contents-of-multiple-.csv-files-into-single-.csv-file.md)
- [env from ConfigMap or Secrets](https://til.duyet.net/data-engineering/env-from-configmap-or-secrets.md)
- [Kubernetes, Helm, Kustomize](https://til.duyet.net/data-engineering/kubernetes.md)
- [initContainer to download file to pod](https://til.duyet.net/data-engineering/kubernetes/initcontainer-to-download-file-to-pod.md)
- [Kustomize: omission of resources](https://til.duyet.net/data-engineering/kubernetes/kustomize-omission-of-resources.md): Skip one of the resources, adding \`$path: delete\` kustomize will skip it.
- [K8S: Services and Labels](https://til.duyet.net/data-engineering/kubernetes/k8s-services-and-labels.md)
- [K8S: PVC stuck in status “Terminating”](https://til.duyet.net/data-engineering/kubernetes/pvc-stuck-in-status-terminating.md)
- [K8S: Port Forward](https://til.duyet.net/data-engineering/kubernetes/k8s-port-forward.md)
- [K8S: Pull an Image from a Private Registry](https://til.duyet.net/data-engineering/kubernetes/pull-an-image-from-a-private-registry.md): Pull Image from Github Docker Package Private
- [Happy helm](https://til.duyet.net/data-engineering/kubernetes/happy-helm.md)
- [Helm: render manifest locally](https://til.duyet.net/data-engineering/kubernetes/helm-render-manifest-locally.md)
- [Helm: Public Helm chart repository with GitHub Pages](https://til.duyet.net/data-engineering/kubernetes/create-public-helm-chart-repository-with-github-pages.md)
- [Apache Airflow](https://til.duyet.net/data-engineering/apache-airflow.md)
- [Extend official Docker image](https://til.duyet.net/data-engineering/apache-airflow/extend-official-docker-image.md)
- [Generate offline SQL upgrade script](https://til.duyet.net/data-engineering/apache-airflow/generate-offline-sql-upgrade-script.md)
- [Airflow in Docker](https://til.duyet.net/data-engineering/apache-airflow/airflow-docker.md)
- [Mastering Airflow UI](https://til.duyet.net/data-engineering/apache-airflow/mastering-airflow-ui.md)
- [Best Practices for Airflow and ETLs](https://til.duyet.net/data-engineering/apache-airflow/best-practices-for-airflow-and-etls.md)
- [Airflow in Docker Compose](https://til.duyet.net/data-engineering/apache-airflow/airflow-in-docker-compose.md): Setup Airflow in Docker Compose
- [Useful SQL queries for Apache Airflow](https://til.duyet.net/data-engineering/apache-airflow/useful-sql-queries-for-apache-airflow.md)
- [Docker](https://til.duyet.net/data-engineering/docker.md)
- ["Distroless" Docker Images](https://til.duyet.net/data-engineering/docker/distroless-docker-images.md)
- [Docker cleanup](https://til.duyet.net/data-engineering/docker/docker-cleanup.md)
- [Optimize the Docker Image Size](https://til.duyet.net/data-engineering/docker/optimize-the-docker-image-size.md)
- [The best Docker base image for Python](https://til.duyet.net/data-engineering/docker/the-best-docker-base-image-for-python.md)
- [Google BigQuery](https://til.duyet.net/db/google-bigquery.md)
- [BigQuery Cancel Running Query](https://til.duyet.net/db/google-bigquery/bigquery-cancel-running-query.md)
- [BigQuery - Split string and get the first part](https://til.duyet.net/db/google-bigquery/bigquery-split-string-and-get-the-first-part.md)
- [BigQuery - UNNEST in SELECT](https://til.duyet.net/db/google-bigquery/bigquery-unnest-in-select.md)
- [Bigquery - Sample queries for audiences based](https://til.duyet.net/db/google-bigquery/bigquery-sample-queries-for-audiences-based.md): These queries return the number of users in the audience. If you'd like to get the list of user IDs in the audience instead, then remove the outermost COUNT() function, e.g., COUNT(DISTINCT user\_id) -
- [BigQuery cookbook for Google Analytics Exported Data](https://til.duyet.net/db/google-bigquery/bigquery-cookbook-for-google-analytics-exported-data.md)
- [Apache Hive](https://til.duyet.net/db/apache-hive.md)
- [Hive - SHOW schemas/tables/create](https://til.duyet.net/db/apache-hive/hive-show-schemas-tables-create.md): These commands will help to show list schemas, list tables and create query of tables.
- [AWS Redshift / Postgres](https://til.duyet.net/db/aws-redshift-postgres.md)
- [Amazon Redshift Utilities](https://til.duyet.net/db/aws-redshift-postgres/amazon-redshift-utilities.md)
- [Postgres - Index Summary](https://til.duyet.net/db/aws-redshift-postgres/postgres-index-summary.md)
- [Postgres - List tables](https://til.duyet.net/db/aws-redshift-postgres/postgres-list-tables.md)
- [Redshift - GRANT](https://til.duyet.net/db/aws-redshift-postgres/redshift-grant.md)
- [Redshift - tables and their owners](https://til.duyet.net/db/aws-redshift-postgres/redshift-tables-and-their-owners.md)
- [Redshift - Check the table size](https://til.duyet.net/db/aws-redshift-postgres/redshift-check-the-table-size.md): Redshift or Postgres is ok
- [Presto](https://til.duyet.net/db/presto.md)
- [Aliyun Data Lake Analytics (Presto) - Add partition on non-existing location](https://til.duyet.net/db/presto/aliyun-data-lake-analytics-presto-add-partition-on-non-existing-location.md)
- [AWS Athena - Add Partition](https://til.duyet.net/db/presto/athena-add-partition.md)
- [Rust](https://til.duyet.net/programming/rust.md)
- [cheats.rs](https://til.duyet.net/programming/rust/cheats.rs.md)
- [Imperative vs Declarative](https://til.duyet.net/programming/rust/imperative-vs-declarative.md)
- [Generate Struct from JSON](https://til.duyet.net/programming/rust/generate-struct-from-json.md)
- [Python](https://til.duyet.net/programming/python.md)
- [YAML config file with environment variables](https://til.duyet.net/programming/python/yaml-config-file-with-environment-variables.md)
- [date\_range\_generator](https://til.duyet.net/programming/python/date_range_generator.md)
- [get\_all\_s3\_keys](https://til.duyet.net/programming/python/get_all_s3_keys.md): Get all S3 Keys by Prefix by Boto3
- [Pipenv](https://til.duyet.net/programming/python/pipenv.md)
- [Golang](https://til.duyet.net/programming/golang.md)
- [Functions and Methods in Go?](https://til.duyet.net/programming/golang/functions-and-methods-in-go.md)
- [Convert JSON to Go struct](https://til.duyet.net/programming/golang/convert-json-to-go-struct.md)
- [Javascript / Typescript](https://til.duyet.net/programming/javascript-typescript.md)
- [Intl.ListFormat](https://til.duyet.net/programming/javascript-typescript/intl.listformat.md): Convert arrays to human-readable lists
- [FE / React](https://til.duyet.net/programming/fe-react.md)
- [Beautiful icons, images, ..](https://til.duyet.net/programming/fe-react/beautiful-icons-images-...md)
- [Flexbox](https://til.duyet.net/programming/fe-react/flexbox.md)
- [Create hook to inject JS script](https://til.duyet.net/programming/fe-react/create-hook-to-inject-js-script.md)
- [Git - Pretty git branch graphs](https://til.duyet.net/unix/git-pretty-git-branch-graphs.md)
- [Checking files in Docker build context](https://til.duyet.net/unix/checking-files-in-docker-build-context.md): Sometimes we need to check files in Docker build context, or just to check .dockerignore is work or not
- [Bash get the directory of the current script](https://til.duyet.net/unix/bash-get-the-directory-of-the-current-script.md)
- [Vim](https://til.duyet.net/unix/vim.md): I switched to use Vim!
- [Find and replace](https://til.duyet.net/unix/find-and-replace.md)
- [\[Fig\] Single machine and distributed system structure](https://til.duyet.net/miscellaneous/fig-single-machine-and-distributed-system-structure.md)
- [Deploying Machine Learning Models at Scale](https://til.duyet.net/miscellaneous/deploying-machine-learning-models-at-scale.md): https://algorithmia.com/blog/deploying-machine-learning-at-scale
- [Bypass a Chrome certificate/HSTS error](https://til.duyet.net/miscellaneous/bypass-a-chrome-certificate-hsts-error.md)
- [Articles](https://til.duyet.net/miscellaneous/articles.md)