SYNC_PRODUCT

P4

3. Wait unitl all the threads completed 

P3

P1

Main Thread

VP3

···

P2

7. Loop the products 

8. Wait unitl all the threads completed 

9. Create and upload reprot to S3

P.N

Thread_GetNextPage

Thread_CreatePeport

1. Get product data from PIM

Loop the pages

ThreadPoolExecutor

1 ··· N

VP4

VP2

6. Get product data from the url fetched from step 5

N<=100

4. Get the next page product data

VP1

5. Wait unit this thread completed

VP.N

ThreadPoolExecutor2

2. Loop the products