Skip to content

tracebloc data pipeline for training/test dataset setup

Notifications You must be signed in to change notification settings

tracebloc/data-ingestors

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Steps to Run Project

To run this project follow these steps:

  • Clone Repository
git clone https://github.com/tracebloc/data-ingestors.git
  • Check required directory inside "/data/shared/" and files are present
    • input_images : containing all images
    • label.csv
  • Create required directory
    • raw_images
    • processed_images
  • Activate Python Environment
  • Move to repository
cd data-ingestors/csv-ingestor
  • Run command to build docker image
docker build -t IMAGE_NAME:TAG PATH_TO_DOCKERFILE .
  • Run command to push docker image
docker push IMAGE_NAME:TAG PATH_TO_DOCKERFILE
  • Run command to start ingestor deployment "deployment.yaml"
kubectl apply -f deployment.yaml
  • Move all images from input_images to raw_images
mv input_images/* raw_images