Data Engineering Tools

Some important tools required for Data Engineering projects.

Start all services

./start.sh

Stop all services

./stop.sh

Restart all services

./restart.sh

Services

• HDFS Namenode - http://0.0.0.0:32763

• Hive Server - http://0.0.0.0:32759/

• Hue - http://0.0.0.0:32762

• Spark - http://0.0.0.0:32766/

  • Worker - http://0.0.0.0:32764/

• Airflow - http://0.0.0.0:8080/

• Flink - http://0.0.0.0:8081

• Livy - http://0.0.0.0:32758

• Adminer - http://0.0.0.0:32767/

• PostgreSQL - 0.0.0.0:32769

• Kafka - 0.0.0.0:9092, 0.0.0.0:9093