NUHMAN.com

Need to process a batch of records from the database, send each to an external API, and update the record's status afterward? This is a common pattern in backend systems — especially for ETL jobs, sync systems, or microservices.

In this guide, we'll:

• Read database records in pages
• Call an API for each record
• Update the status based on the response
• Schedule it to run periodically

🏗️ 1. Entity Layer

java

import jakarta.persistence.*;

@Entity
public class YourEntity {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String name;
    private String status; // e.g., "Pending", "Processed", "Failed"

    // Getters and Setters
}

📦 2. Repository Layer

java

import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface YourEntityRepository extends JpaRepository<YourEntity, Long> {
}

🔁 3. Batch Processing Service

java

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.*;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.data.domain.*;

import java.util.List;

@Service
public class BatchProcessingService {

    @Autowired
    private YourEntityRepository yourEntityRepository;

    @Autowired
    private RestTemplate restTemplate;

    @Transactional
    public void processBatch(int batchSize) {
        Pageable pageable = PageRequest.of(0, batchSize);
        Page<YourEntity> page;

        do {
            page = yourEntityRepository.findAll(pageable);
            List<YourEntity> entities = page.getContent();

            for (YourEntity entity : entities) {
                try {
                    HttpEntity<YourEntity> request = new HttpEntity<>(entity);
                    ResponseEntity<String> response = restTemplate.postForEntity("http://your-api-endpoint", request, String.class);

                    if (response.getStatusCode().is2xxSuccessful()) {
                        entity.setStatus("Processed");
                    } else {
                        entity.setStatus("Failed");
                    }
                } catch (Exception e) {
                    entity.setStatus("Failed");
                    // log the error
                }
                yourEntityRepository.save(entity);
            }

            pageable = page.nextPageable();
        } while (page.hasNext());
    }
}

⏰ 4. Scheduled Runner

java

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;

@Service
public class ScheduledBatchRunner {

    @Autowired
    private BatchProcessingService batchProcessingService;

    @Scheduled(fixedRate = 60000)  // Every 60 seconds
    public void runBatch() {
        batchProcessingService.processBatch(100);  // Process 100 records at a time
    }
}

⚙️ 5. App Configuration

java

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestTemplate;

@Configuration
public class AppConfig {

    @Bean
    public RestTemplate restTemplate() {
        return new RestTemplate();
    }
}

🧠 Key Concepts:

• Pagination with Spring Data JPA: Avoids loading all records into memory.
• RestTemplate for API calls: Use WebClient if you want non-blocking calls.
• Transactional scope: Ensures each page/record is properly committed.
• Status tracking: Makes the process idempotent and observable.
• @Scheduled: Automates periodic execution.

✅ Tips:

• Use a separate column like lastUpdatedAt or retryCount if needed.
• Add logging to track progress and failures.
• Consider retry mechanisms for transient API failures.
• If dealing with millions of rows, consider using Spring Batch or chunking via custom queries.