# Data Engineering

- [AWS](/data-engineering/aws.md)
- [Make an NVMe volume available for use on AWS EC2](/data-engineering/aws/make-an-nvme-volume-available-for-use-on-aws-ec2.md)
- [AWS EMR](/data-engineering/aws/emr.md): Manage EMR Cluster
- [S3 Dist CP](/data-engineering/aws/emr/s3-dist-cp.md): Using S3DistCp, you can efficiently copy large amounts of data from Amazon S3/HDFS into S3/HDFS
- [Manage services](/data-engineering/aws/emr/manage-services.md)
- [EMR - Tez](/data-engineering/aws/emr/emr-tez.md): Upscale/Downscale of the emr cluster corrupts hdfs jars /apps/tez/tez.tar.gz
- [Issues](/data-engineering/aws/emr/issues.md)
- [Tools](/data-engineering/tools.md)
- [Shell](/data-engineering/shell.md)
- [MacOS's Touch ID on Terminal](/data-engineering/shell/macoss-touch-id-on-terminal.md)
- [Using \`sed\` to find and replace in file](/data-engineering/shell/using-sed-to-find-and-replace-in-file.md): sed = Stream EDitor
- [Merging contents of multiple .csv files into single .csv file](/data-engineering/shell/merging-contents-of-multiple-.csv-files-into-single-.csv-file.md)
- [env from ConfigMap or Secrets](/data-engineering/env-from-configmap-or-secrets.md)
- [Kubernetes, Helm, Kustomize](/data-engineering/kubernetes.md)
- [initContainer to download file to pod](/data-engineering/kubernetes/initcontainer-to-download-file-to-pod.md)
- [Kustomize: omission of resources](/data-engineering/kubernetes/kustomize-omission-of-resources.md): Skip one of the resources, adding \`$path: delete\` kustomize will skip it.
- [K8S: Services and Labels](/data-engineering/kubernetes/k8s-services-and-labels.md)
- [K8S: PVC stuck in status “Terminating”](/data-engineering/kubernetes/pvc-stuck-in-status-terminating.md)
- [K8S: Port Forward](/data-engineering/kubernetes/k8s-port-forward.md)
- [K8S: Pull an Image from a Private Registry](/data-engineering/kubernetes/pull-an-image-from-a-private-registry.md): Pull Image from Github Docker Package Private
- [Happy helm](/data-engineering/kubernetes/happy-helm.md)
- [Helm: render manifest locally](/data-engineering/kubernetes/helm-render-manifest-locally.md)
- [Helm: Public Helm chart repository with GitHub Pages](/data-engineering/kubernetes/create-public-helm-chart-repository-with-github-pages.md)
- [Apache Airflow](/data-engineering/apache-airflow.md)
- [Extend official Docker image](/data-engineering/apache-airflow/extend-official-docker-image.md)
- [Generate offline SQL upgrade script](/data-engineering/apache-airflow/generate-offline-sql-upgrade-script.md)
- [Airflow in Docker](/data-engineering/apache-airflow/airflow-docker.md)
- [Mastering Airflow UI](/data-engineering/apache-airflow/mastering-airflow-ui.md)
- [Best Practices for Airflow and ETLs](/data-engineering/apache-airflow/best-practices-for-airflow-and-etls.md)
- [Airflow in Docker Compose](/data-engineering/apache-airflow/airflow-in-docker-compose.md): Setup Airflow in Docker Compose
- [Useful SQL queries for Apache Airflow](/data-engineering/apache-airflow/useful-sql-queries-for-apache-airflow.md)
- [Docker](/data-engineering/docker.md)
- ["Distroless" Docker Images](/data-engineering/docker/distroless-docker-images.md)
- [Docker cleanup](/data-engineering/docker/docker-cleanup.md)
- [Optimize the Docker Image Size](/data-engineering/docker/optimize-the-docker-image-size.md)
- [The best Docker base image for Python](/data-engineering/docker/the-best-docker-base-image-for-python.md)