Why a Spring Boot Service Becomes Slow Under 1,000 Concurrent Requests?

Introduction

A Spring Boot application may perform exceptionally well during development. An API responds in 30 milliseconds, database queries complete quickly, and CPU utilization remains low. However, once the application is deployed to production and receives hundreds or thousands of concurrent requests, response times can increase dramatically.

An endpoint that previously responded in 50 milliseconds might begin taking 2 seconds. Some requests may time out, database connections may become exhausted, and the application may eventually return HTTP 503 or 504 errors.

A common assumption is that the application needs more CPU, memory, or server instances. While scaling can help, it does not necessarily address the underlying problem.

The real issue is often resource contention, thread blocking, connection pool exhaustion, inefficient database access, or an architectural bottleneck.

Understanding these problems requires looking beyond the number of requests per second and examining how requests consume threads, database connections, memory, CPU time, and downstream service capacity.

In this article, we will explore why Spring Boot services become slow under 1,000 concurrent requests, how to identify the actual bottleneck, and how to build applications that remain responsive under load.

We will use Spring Boot, Spring MVC, embedded Tomcat, JDBC, HikariCP, PostgreSQL, Java concurrency, and production monitoring examples.

1. What Does 1,000 Concurrent Requests Actually Mean?

Before investigating performance problems, it is important to distinguish concurrency from throughput.

• Concurrency: The number of requests that are in progress at the same time.
• Throughput: The number of requests completed per second.
• Latency: The time required to complete a request.
• Saturation: The point at which a resource reaches its effective capacity and additional demand causes queues, timeouts, or failures.

Consider an application that receives 1,000 concurrent requests.

If each request completes in 100 milliseconds, those requests may finish quickly, assuming sufficient capacity.

If each request spends 2 seconds waiting for a database connection, the same concurrency can create a substantial queue and lead to cascading delays.

Concurrency alone does not tell us whether a service can handle the workload.

Concurrency versus throughput

Little’s Law describes the relationship between average concurrency, throughput, and average time spent in a system:

L = λ × W

Where:

• L is the average number of requests in the system.
• λ is the average throughput in requests per second.
• W is the average time spent in the system, in seconds.

For example, if a service sustains 500 requests per second with an average response time of 200 milliseconds:

L = 500 × 0.2 = 100 concurrent requests.

If average latency increases to 2 seconds while throughput remains 500 requests per second:

L = 500 × 2 = 1,000 concurrent requests.

This illustrates an important principle: slow responses cause requests to remain in the system longer, increasing concurrency and putting additional pressure on application resources.

Little’s Law is a relationship between averages in a stable system, not a guarantee that a service can sustain a particular workload.

Why 1,000 requests can be a problem

The impact depends on what each request does.

Request workload	Potential bottleneck
CPU-intensive JSON transformation	CPU saturation
Database-backed REST API	Database connection pool
Slow third-party HTTP call	Outbound connection pool and blocked threads
Large file processing	Heap usage, allocation rate, disk I/O
Complex Hibernate queries	Database execution and ORM overhead
Synchronous processing with long waits	Request-thread exhaustion
Excessive logging	CPU, disk I/O, logging locks

A service serving 1,000 lightweight requests may run comfortably, while another service handling just 100 requests with slow external dependencies may struggle.

The first task is therefore to identify which resource is becoming saturated.

2. How a Spring Boot Request Travels Through the Application

To understand where performance degrades, we first need to understand the request-processing lifecycle.

A typical Spring Boot application using Spring MVC and embedded Tomcat follows this path:

1. The client sends an HTTP request.
2. The operating system and network stack receive the connection and data.
3. Tomcat accepts and processes the connection.
4. A Tomcat worker thread executes the request.
5. Spring’s DispatcherServlet routes the request to the appropriate controller.
6. The service layer executes business logic.
7. The application accesses the database, cache, or another service.
8. The response is serialized into JSON and sent back to the client.

Every stage can introduce latency.

For example, a request might spend only 20 milliseconds executing Java code but another 800 milliseconds waiting for a database connection and 1.2 seconds waiting for a downstream service.

Optimizing the Java code will not solve the main problem if most of the response time is spent waiting elsewhere.

Request lifecycle diagram

The critical point is that the Tomcat request thread may remain occupied while the application waits for a database, an HTTP response, a lock, or another blocking operation.

If enough threads become occupied, incoming requests must wait for an available worker.

3. Root Cause #1: Tomcat Request Thread Pool Exhaustion

One of the most common reasons a Spring MVC application becomes slow under concurrent load is request-thread exhaustion.

Spring Boot applications using Spring MVC typically run on embedded Tomcat unless another server is configured.

Tomcat processes application requests using a configurable worker-thread pool. When a request executes blocking operations, its worker thread generally remains occupied until that work finishes.

Consider the following configuration:

server:
  tomcat:
    threads:
      max: 200
      min-spare: 20

Here, Tomcat is configured with a maximum of 200 worker threads.

This does not mean the application can handle only 200 concurrent clients. Keep-alive connections can exist without continuously occupying a worker thread, and actual concurrency depends on the workload.

However, if 200 requests simultaneously occupy all available worker threads, additional requests that require a worker may have to wait.

Example: A slow downstream service

Consider a controller that calls an external service synchronously.

@RestController
@RequestMapping("/api/orders")
public class OrderController {
    private final OrderService orderService;

    public OrderController(OrderService orderService) {
        this.orderService = orderService;
    }

    @GetMapping("/{id}")
    public Order getOrder(@PathVariable Long id) {
        return orderService.getOrder(id);
    }
}

The service implementation might look like this:

@Service
public class OrderService {
    private final RestTemplate restTemplate;

    public OrderService(RestTemplate restTemplate) {
        this.restTemplate = restTemplate;
    }

    public Order getOrder(Long id) {
        Order order = loadOrder(id);

        PaymentStatus status = restTemplate.getForObject(
            "http://payment-service/status/" + id,
            PaymentStatus.class
        );

        order.setPaymentStatus(status);

        return order;
    }

    private Order loadOrder(Long id) {
        // Database access omitted
        return new Order(id, "PROCESSING");
    }
}

Suppose the payment service takes 2 seconds to respond.

If 200 Tomcat worker threads are simultaneously waiting for that service, the application may have no worker threads available for new requests.

The database might be healthy, and CPU utilization might remain moderate, yet response times can continue increasing.

How to identify thread exhaustion

Monitor the following metrics:

• tomcat.threads.busy
• tomcat.threads.current
• tomcat.threads.config.max
• HTTP request latency
• HTTP request queueing and connection errors
• Thread states in Java thread dumps

For a typical Spring Boot application with Actuator and Micrometer, these metrics may be available through the /actuator/metrics endpoint.

For example:

curl http://localhost:8080/actuator/metrics/tomcat.threads.busy

The exact availability and naming of metrics depend on your Spring Boot version, Micrometer registry, and server configuration.

If the busy thread count remains near the configured maximum while request latency rises, investigate what those threads are doing before increasing the limit.

How to fix the problem

1. Reduce blocking time. Optimize database queries, configure HTTP timeouts, and eliminate unnecessary synchronous calls.

2. Use bulkheads. Limit the number of requests allowed to call a slow downstream dependency simultaneously.

3. Introduce bounded asynchronous processing where appropriate. This can reduce request-thread occupancy, but asynchronous work still needs bounded queues and sufficient downstream capacity.

4. Tune Tomcat only after measuring. More threads can help when the workload is mostly waiting on I/O and downstream systems have spare capacity. More threads can also increase memory consumption, context switching, and pressure on already-saturated dependencies.

Important: Increasing Tomcat’s maximum threads from 200 to 800 does not automatically improve performance. If the database can support only 50 active operations efficiently, the additional threads may simply create a larger queue at the database.

4. Root Cause #2: Database Connection Pool Exhaustion

Database connection pool exhaustion is another major cause of slow Spring Boot APIs.

Spring Boot applications commonly use HikariCP for JDBC connection pooling.

Creating a database connection is relatively expensive compared with reusing an existing connection. A pool maintains reusable connections so application requests can acquire them when needed.

Consider this configuration:

spring:
  datasource:
    url: jdbc:postgresql://localhost:5432/orders
    username: app_user
    password: ${DB_PASSWORD}
    hikari:
      maximum-pool-size: 20
      minimum-idle: 5
      connection-timeout: 3000

The pool permits at most 20 connections.

If 100 concurrent requests all need database access at approximately the same time, no more than 20 can hold a pooled connection simultaneously. Other requests may wait until a connection becomes available.

If connections are held for too long, the waiting queue grows.

Example: Connection pool waiting

Assume:

• Concurrent requests: 1,000
• Database connection pool size: 20
• Average connection hold time: 100 milliseconds

Under idealized conditions, if every operation uses exactly one connection for 100 milliseconds and the database can sustain the workload, the pool’s approximate maximum throughput for those operations is:

Throughput ≈ Pool size / Connection hold time

Throughput ≈ 20 / 0.1 = 200 operations per second.

This is a theoretical approximation, not a production capacity guarantee. Real throughput depends on transaction duration, query complexity, database capacity, scheduling, and contention.

If a connection is held for 500 milliseconds instead, the same rough calculation becomes:

Throughput ≈ 20 / 0.5 = 40 operations per second.

The pool size has not changed, but its potential throughput has dropped by a factor of five.

Common causes of connection pool exhaustion

Long-running SQL queries

Queries that perform full table scans, expensive joins, or poorly selective filters can hold connections longer than expected.

Long-running transactions

A transaction may acquire a connection and keep it until the transaction finishes. Unnecessary work performed inside the transaction increases connection occupancy.

External API calls inside transactions

Consider this pattern:

@Transactional
public Order createOrder(CreateOrderRequest request) {
    Order order = orderRepository.save(
        new Order(request.customerId())
    );

    paymentClient.charge(request.paymentDetails());

    notificationClient.sendOrderCreated(order.getId());

    return order;
}

The transaction may remain open while the application waits for the payment service and notification service.

This increases the time a database connection is held and may also keep database locks active longer than necessary.

A better design is to keep the database transaction focused on database work and handle remote operations outside that transaction when business consistency permits. For critical workflows, consider an outbox pattern, saga, or explicit state machine rather than simply moving calls without considering failure handling.

Connection leaks

Connections can be leaked when manually acquired resources are not closed properly. Standard Spring-managed JDBC operations and transaction management usually handle connection lifecycle automatically, but custom JDBC code must release resources correctly.

How to identify connection pool exhaustion

Monitor HikariCP metrics such as:

• Active connections
• Idle connections
• Maximum connections
• Pending connection acquisitions
• Connection acquisition time
• Connection timeout count

Common Micrometer metric names include:

hikaricp.connections.active
hikaricp.connections.idle
hikaricp.connections.max
hikaricp.connections.pending
hikaricp.connections.acquire
hikaricp.connections.timeout

Metric availability and exact names depend on the registry and version.

When active connections stay near the maximum and pending acquisitions increase, investigate connection occupancy and database behavior.

How to fix database connection pool issues

1. Optimize slow SQL queries using EXPLAIN ANALYZE where appropriate.
2. Add indexes that match actual query patterns.
3. Keep transactions short.
4. Avoid remote network calls inside database transactions.
5. Check for connection leaks and unexpectedly long connection hold times.
6. Tune the connection pool based on database capacity and measured demand.
7. Consider caching frequently accessed, relatively stable data.
8. Use pagination and limits instead of loading unbounded result sets.

Production lesson: A large connection pool is not always faster. Too many simultaneous database operations can increase lock contention, memory consumption, context switching, and query latency.

5. Root Cause #3: Slow SQL Queries and Database Contention

Even if HikariCP has enough connections, the database itself can become the bottleneck.

A Spring Boot application might have sufficient Tomcat threads and a healthy connection pool but still experience high latency because the database cannot execute queries quickly enough.

Consider a typical query:

SELECT id, customer_id, status, created_at
FROM orders
WHERE customer_id = ?
ORDER BY created_at DESC
LIMIT 20;

If the table contains millions of records and the database lacks a useful index, the query may scan and sort a large number of rows.

Under low load, the query might appear acceptable. Under high concurrency, repeated scans and sorts can increase CPU usage, disk I/O, and memory pressure.

Indexing the right columns

A possible index for this access pattern is:

CREATE INDEX idx_orders_customer_created
ON orders (customer_id, created_at DESC);

Whether this index improves performance depends on the database engine, query plan, table distribution, and existing indexes.

Validate the execution plan rather than creating indexes indiscriminately.

For PostgreSQL, inspect a representative query using:

EXPLAIN (ANALYZE, BUFFERS)
SELECT id, customer_id, status, created_at
FROM orders
WHERE customer_id = 123
ORDER BY created_at DESC
LIMIT 20;

EXPLAIN ANALYZE executes the query. Use caution when analyzing expensive queries in production, and remember that its timing may differ from normal application execution.

Other database bottlenecks

• Lock contention between transactions
• Deadlocks and transaction retries
• Missing or inefficient indexes
• Excessive joins
• Large result sets
• Inefficient pagination
• Database CPU saturation
• Disk I/O bottlenecks
• Poor query plans caused by stale statistics
• Connection limits or workload-management limits

A practical debugging approach

Measure database query latency separately from the full HTTP request.

For example:

Measurement	What it helps identify
HTTP latency	Total user-visible delay
Connection acquisition time	Pool waiting
SQL execution time	Query and database delay
Transaction duration	Overall transaction occupancy
Rows returned	Excessive data retrieval
Database CPU and I/O	Database resource saturation

Do not assume that slow HTTP requests necessarily mean slow SQL execution. Connection acquisition, transaction overhead, ORM processing, and downstream calls can contribute substantial additional latency.

6. Root Cause #4: Blocking External API Calls

Modern Spring Boot applications rarely operate in isolation. They may call payment gateways, authentication services, customer APIs, inventory services, notification platforms, or other microservices.

These dependencies can become bottlenecks even when the Spring Boot application itself is healthy.

Consider a checkout API that performs three sequential operations:

public CheckoutResponse checkout(CheckoutRequest request) {
    Customer customer = customerClient.getCustomer(
        request.customerId()
    );

    Inventory inventory = inventoryClient.checkAvailability(
        request.productId()
    );

    PaymentResult payment = paymentClient.processPayment(
        request.paymentDetails()
    );

    return new CheckoutResponse(
        customer.id(),
        inventory.available(),
        payment.status()
    );
}

Suppose the downstream services have the following average latencies:

Operation	Average latency
Customer API	150 ms
Inventory API	200 ms
Payment API	500 ms
Application processing	20 ms
Total	870 ms

Because the operations are sequential, the total is approximately the sum of their individual latencies.

If 1,000 requests execute this flow concurrently, the application can generate substantial pressure on all three downstream services.

The payment API may become saturated first. Its response times increase, Spring Boot request threads remain occupied longer, and the application’s own queues begin to grow.

Configure connection and response timeouts

Never depend on unbounded waits for external services.

For example, using Spring’s RestClient with a JDK HTTP client:

@Bean
RestClient paymentRestClient() {
    HttpClient httpClient = HttpClient.newBuilder()
        .connectTimeout(Duration.ofSeconds(1))
        .build();

    JdkClientHttpRequestFactory requestFactory =
        new JdkClientHttpRequestFactory(httpClient);

    requestFactory.setReadTimeout(Duration.ofSeconds(2));

    return RestClient.builder()
        .requestFactory(requestFactory)
        .baseUrl("http://payment-service")
        .build();
}

This illustrates explicit connection and read timeout configuration. Exact APIs and supported timeout settings vary by Spring Framework version and HTTP request factory.

A connection timeout controls how long connection establishment may take. A read timeout limits waiting for response data according to the client’s semantics; it is not necessarily a strict end-to-end deadline for the entire request.

Production applications should also consider connection-pool acquisition timeouts, total request deadlines, cancellation behavior, and the timeout configuration of proxies and load balancers.

Use circuit breakers

A circuit breaker prevents an application from repeatedly calling a dependency that is already failing or too slow.

The typical states are:

• Closed: Calls are allowed, and failures are monitored.
• Open: Calls are rejected or handled through a fallback.
• Half-open: A limited number of trial calls determine whether the dependency has recovered.

Spring applications commonly use Resilience4j for circuit breakers, time limiters, retries, and bulkheads.

Illustrative configuration:

resilience4j:
  circuitbreaker:
    instances:
      paymentService:
        slidingWindowSize: 20
        minimumNumberOfCalls: 10
        failureRateThreshold: 50
        waitDurationInOpenState: 10s
  timelimiter:
    instances:
      paymentService:
        timeoutDuration: 2s

This is a starting example, not a universal production configuration. Circuit-breaker thresholds should reflect observed latency, error rates, traffic volume, and recovery behavior.

A circuit breaker does not make a dependency faster. It limits the damage that an unhealthy dependency can cause.

Avoid uncontrolled retries

Retries can improve reliability when failures are transient, but they can make an overloaded service even slower.

Suppose 1,000 requests arrive and each request retries a failed downstream operation twice. The downstream system may receive as many as 3,000 attempts for that operation, assuming every original request makes both retries.

Use bounded retries, exponential backoff, jitter, and an overall deadline. Retry only when the operation is safe to repeat or uses appropriate idempotency controls.

For payment processing and other state-changing operations, a timeout does not necessarily mean the operation failed. The server may have completed the operation while the response was lost.

7. Root Cause #5: Inefficient Thread Pool Configuration

Thread pools are essential to controlling concurrency, but poor configuration can introduce bottlenecks.

Spring Boot applications may use several independent pools:

• Tomcat request threads
• HikariCP database connections
• HTTP client connection pools
• ThreadPoolTaskExecutor for asynchronous tasks
• ForkJoinPool for parallel operations
• Scheduler threads
• Message-consumer threads

Each pool has a different purpose and capacity.

Increasing one pool without considering the others may simply transfer the queue to another resource.

Example: An asynchronous executor with an unbounded queue

@Bean
Executor taskExecutor() {
    ThreadPoolTaskExecutor executor =
        new ThreadPoolTaskExecutor();

    executor.setCorePoolSize(20);
    executor.setMaxPoolSize(100);
    executor.setQueueCapacity(1000);
    executor.setThreadNamePrefix("async-");

    executor.initialize();
    return executor;
}

This configuration creates a bounded queue of 1,000 tasks, which is preferable to an unbounded queue for overload protection. However, its behavior needs to be understood.

With a typical ThreadPoolExecutor policy, tasks are first submitted to the core workers. Additional tasks are queued until the queue is full. Only then does the executor expand beyond its core size, up to its maximum pool size.

A large queue may therefore increase waiting time instead of increasing immediate processing capacity.

For latency-sensitive operations, a queue that is too large can allow requests to wait so long that their results are no longer useful.

Choose queue sizes based on deadlines

Consider an asynchronous task that has a 500-millisecond end-to-end deadline.

If it waits 2 seconds in an executor queue before starting, it has already missed its deadline before doing any work.

Useful safeguards include:

• Bounded queues
• Explicit rejection policies
• Queue-wait-time metrics
• Task deadlines
• Cancellation where supported
• Separate executors for independent workloads
• Graceful overload responses

An executor is not a substitute for capacity planning.

CPU-bound versus I/O-bound work

CPU-bound tasks, such as cryptographic processing or expensive transformations, compete for CPU time. Creating substantially more worker threads than available CPU capacity may increase context switching without improving throughput.

I/O-bound tasks spend time waiting on databases, networks, or files. They may benefit from a different concurrency strategy, but their throughput is still constrained by downstream resources.

Choose thread counts based on measured workload behavior rather than a fixed rule such as “always use twice the number of CPU cores.”

8. Root Cause #6: Excessive Object Allocation and Garbage Collection

Java applications create objects continuously. Under heavy traffic, object allocation can become a significant performance factor.

A request may allocate:

• Request and response DTOs
• JSON parser and serializer objects
• Collections and intermediate data structures
• Hibernate entities and proxies
• Strings and byte arrays
• Logging arguments
• Cache entries
• Temporary objects created during transformations

If allocation rates become high, the JVM must spend more time reclaiming memory.

What happens during garbage collection?

The JVM divides heap memory into regions or generations depending on the selected garbage collector and its implementation.

In a generational collector, short-lived objects are commonly reclaimed efficiently, while longer-lived objects may survive multiple collection cycles.

However, large allocations, excessive live data, promotion pressure, or insufficient heap headroom can increase collection overhead.

Potential symptoms include:

• Increased garbage-collection frequency
• Longer GC pauses
• High allocation rates
• Repeated old-generation collections
• Heap usage that rises and falls sharply
• Increased request latency during GC activity

Modern garbage collectors such as G1 are designed to reduce pause times, but they cannot eliminate the cost of excessive allocation or a heap dominated by live objects.

Example: Loading too much data

Consider this repository method:

public List<OrderResponse> getAllOrders() {
    return orderRepository.findAll()
        .stream()
        .map(this::toResponse)
        .toList();
}

If the table contains millions of records, returning everything in one request can create enormous database results, ORM object graphs, temporary collections, serialized responses, and network payloads.

The application may run out of heap memory or spend excessive time in garbage collection.

A better approach is to retrieve only the required records:

public Page<OrderResponse> getOrders(Pageable pageable) {
    return orderRepository.findAll(pageable)
        .map(this::toResponse);
}

Example controller:

@GetMapping
public Page<OrderResponse> getOrders(
        @PageableDefault(size = 20, sort = "createdAt",
                         direction = Sort.Direction.DESC)
        Pageable pageable) {

    return orderService.getOrders(pageable);
}

Pagination limits the amount of data processed per response. For large datasets, keyset pagination may perform better than offset pagination, depending on the query and index design.

How to diagnose GC problems

Useful tools include:

• Java Flight Recorder (JFR)
• Java Mission Control (JMC)
• GC logs
• Heap dumps
• Eclipse Memory Analyzer (MAT)
• JVM metrics exported through Micrometer

For a modern JVM, GC logging can be enabled with options such as:

-Xlog:gc*,safepoint:file=logs/gc.log:time,uptime,level,tags

The exact JVM flags and output should be validated against the Java version in use.

GC logs can help correlate pauses, heap occupancy, allocation pressure, and latency spikes. A heap dump is particularly useful when investigating retained objects or memory leaks, but capturing one in production can have substantial performance and disk-space implications.

Important: High heap usage alone does not prove that garbage collection is the root cause. Focus on allocation rate, pause time, post-GC live-set size, and their correlation with application latency.

9. Root Cause #7: CPU Saturation and Expensive Business Logic

Some endpoints perform substantial computation before returning a response.

Examples include:

• Large JSON transformations
• Complex validation rules
• Encryption and decryption
• Compression
• Report generation
• Data aggregation
• Regular-expression processing
• AI inference or embedding generation
• Large in-memory sorting operations

When CPU demand exceeds available processing capacity, runnable threads compete for CPU time.

Response times increase even when database connections are available.

Example: An expensive operation inside the request path

@GetMapping("/report")
public ReportResponse generateReport() {
    return reportService.generateLargeReport();
}

If generating a report requires scanning and aggregating millions of records, performing the work synchronously on a request thread can increase both CPU consumption and request-thread occupancy.

Possible improvements include:

1. Optimize algorithms and data structures.
2. Push appropriate aggregations to the database.
3. Cache reusable computed results.
4. Process large reports asynchronously.
5. Move expensive workloads to a separate worker service.
6. Apply concurrency limits to CPU-intensive operations.

For tasks that do not need to complete before the client receives a response, an asynchronous job model can be more appropriate than a long-running HTTP request.

For example, the API can submit a report-generation job and return HTTP 202 Accepted with a job identifier. The client can check the job status later.

This design reduces the time an HTTP request remains active, but it does not eliminate the computation cost. The background workers still need capacity limits, durable job handling where required, and monitoring.

10. Root Cause #8: N+1 Queries and Hibernate Performance Problems

Spring Boot applications using Spring Data JPA and Hibernate can suffer from the N+1 query problem.

Suppose an endpoint retrieves 100 orders and then accesses the customer associated with each order.

List<Order> orders = orderRepository.findAll();

return orders.stream()
    .map(order -> new OrderResponse(
        order.getId(),
        order.getCustomer().getName()
    ))
    .toList();

Depending on the entity mappings, fetch strategy, and persistence context, the application may execute one query to retrieve orders and additional queries to load associated customers.

For 100 orders, this could result in approximately 101 queries rather than one or a small number of efficient queries.

At low traffic, the problem may be difficult to notice. At high concurrency, the additional queries can overwhelm the database and connection pool.

Solutions for N+1 queries

Use a fetch join where appropriate:

@Query("""
    SELECT o
    FROM Order o
    JOIN FETCH o.customer
    WHERE o.status = :status
    """)
List<Order> findOrdersWithCustomer(
    @Param("status") OrderStatus status
);

Use DTO projections to fetch only the columns needed for the response.

Use entity graphs when a specific query needs selected associations loaded.

Avoid blindly enabling eager fetching. Loading every associated entity can produce large joins, duplicated rows, and unnecessary memory consumption.

Also be careful with pagination over collection fetch joins, because Hibernate may need to process more data than expected or apply pagination in memory in some configurations.

The correct solution depends on the entity model and access pattern. Measure SQL query counts and query duration to confirm that the change actually improves performance.

11. Root Cause #9: Locks and Shared Mutable State

Concurrency bugs can become performance problems even when the database and CPU are not fully utilized.

Consider a service that synchronizes access to a shared object:

@Service
public class StatisticsService {
    private final Map<String, Long> statistics =
        new HashMap<>();

    public synchronized void record(String key) {
        statistics.merge(key, 1L, Long::sum);
    }
}

The synchronized method allows only one thread at a time to execute record() on the same service instance.

If every request needs to update this shared statistics object, requests may queue behind the same monitor.

The critical section is small, but under high contention the lock can limit throughput.

Common sources of contention

• Synchronized methods and blocks
• Shared mutable collections
• Global caches with coarse-grained locking
• Session or transaction locks
• Database row locks
• Distributed locks
• Contended counters
• Blocking queues with slow consumers

How to diagnose lock contention

Take multiple thread dumps while the application is slow and inspect whether many threads are in BLOCKED state or waiting on the same monitor.

Java Flight Recorder can also help identify monitor contention, thread parking, and time spent waiting.

A single thread dump is a snapshot, so multiple captures are more useful for distinguishing persistent contention from a short-lived event.

How to improve concurrency

• Minimize the time spent inside synchronized sections.
• Avoid performing I/O while holding locks.
• Use appropriate concurrent data structures.
• Partition shared state when possible.
• Use atomic operations for suitable simple updates.
• Avoid introducing complex lock-free algorithms without evidence that they are needed.

Replacing a synchronized block with ConcurrentHashMap is not automatically sufficient. For example, a single highly contended key may still limit throughput.

The objective is to remove unnecessary serialization while preserving correctness.

12. Root Cause #10: Synchronous Processing Where Asynchronous Processing Would Help

Spring MVC is a synchronous request-processing model by default. This is not inherently a problem: Spring MVC is widely used in high-throughput production systems.

The issue arises when request threads spend significant time waiting for operations that could be handled more efficiently using asynchronous I/O or background processing.

For example, consider an API that retrieves customer data and then waits for several independent services.

Customer customer = customerClient.getCustomer(id);
List<Product> products = productClient.getProducts(id);
List<Offer> offers = offerClient.getOffers(id);

If these operations are independent, executing them sequentially may waste time.

A concurrent approach may reduce total latency by overlapping I/O, provided that the downstream services can safely handle the additional concurrency.

Example using CompletableFuture

CompletableFuture<Customer> customerFuture =
    CompletableFuture.supplyAsync(
        () -> customerClient.getCustomer(id),
        ioExecutor
    );

CompletableFuture<List<Product>> productsFuture =
    CompletableFuture.supplyAsync(
        () -> productClient.getProducts(id),
        ioExecutor
    );

CompletableFuture<List<Offer>> offersFuture =
    CompletableFuture.supplyAsync(
        () -> offerClient.getOffers(id),
        ioExecutor
    );

return customerFuture
    .thenCombine(productsFuture, CustomerProducts::new)
    .thenCombine(offersFuture, this::buildResponse)
    .join();

This example illustrates concurrent execution using an explicitly configured executor. The helper types and methods are illustrative.

Important: CompletableFuture does not automatically make blocking calls non-blocking. If each supplier blocks on network I/O, it occupies a worker thread in ioExecutor while waiting. The final join() also blocks the calling thread until the result is available.

The example can reduce total elapsed time by parallelizing independent operations, but it does not necessarily reduce thread consumption.

For genuinely non-blocking I/O, consider Spring WebFlux with a compatible reactive HTTP client, such as WebClient, and a reactive end-to-end pipeline. JDBC and many traditional database drivers remain blocking, so combining reactive HTTP with blocking JDBC requires careful scheduling and capacity management.

When asynchronous processing makes sense

• Independent downstream I/O calls can execute concurrently.
• A request triggers work that does not need to finish before returning.
• A compatible non-blocking stack can reduce the number of threads waiting on network operations.
• Work can be queued with explicit limits and processed under controlled concurrency.

When it may not help

• The database is already saturated.
• The downstream service cannot support more concurrent calls.
• The work is CPU-bound and there are insufficient CPU resources.
• The task requires strict sequential consistency.
• An unbounded executor queue merely moves the overload elsewhere.

Asynchronous execution changes how work is scheduled. It does not remove the need to control how much work the system accepts.

13. Root Cause #11: Excessive Logging and JSON Serialization

Logging and serialization are often overlooked during performance investigations.

A request may generate multiple log entries, serialize a large response, or write large request and response bodies to logs.

Under high concurrency, these operations can consume significant CPU and memory bandwidth.

Expensive logging example

log.info("Request payload: {}", objectMapper.writeValueAsString(request));

This serializes the request even when the log level may not require that output, because the serialization call is evaluated before the logging method is invoked.

A safer approach is:

if (log.isDebugEnabled()) {
    log.debug(
        "Request payload: {}",
        objectMapper.writeValueAsString(request)
    );
}

However, request-body logging should be limited even when debug logging is enabled. Sensitive data, large payloads, and high-volume logs can introduce security, storage, and performance concerns.

Other potential improvements include:

• Avoid logging entire entities and payloads.
• Prefer structured, compact logs.
• Sample high-volume diagnostic events where appropriate.
• Measure asynchronous logging queue behavior.
• Avoid logging synchronously to slow remote destinations.
• Profile JSON serialization and response sizes.

A high-throughput API should log meaningful operational events without turning every request into a large logging workload.

14. Root Cause #12: Cache Misses and Cache Stampedes

Caching can reduce database load, but poor cache behavior can create performance problems.

Suppose an endpoint normally retrieves customer details from Hazelcast or Redis.

Under normal conditions, the cache serves most requests quickly. However, if a popular cache entry expires and many requests attempt to reload it simultaneously, all of them may query the database.

This is known as a cache stampede or cache avalanche pattern, depending on the circumstances.

Example

public Customer getCustomer(Long customerId) {
    String key = "customer:" + customerId;

    Customer customer = cache.get(key);

    if (customer != null) {
        return customer;
    }

    customer = customerRepository.findById(customerId)
        .orElseThrow();

    cache.put(key, customer);

    return customer;
}

The example has a race condition at the application level: multiple requests can miss the same key and independently execute the database query before any request populates the cache.

Potential mitigations include:

• Per-key request coalescing or single-flight loading
• Appropriate TTL jitter to avoid synchronized expirations
• Controlled background refresh
• Cache warming for predictable traffic spikes
• Stale-while-revalidate strategies when business requirements allow them
• Limits on concurrent cache-miss loading

Distributed caches also introduce their own network latency, serialization costs, and failure modes. A cache should be measured as part of the request path rather than assumed to be free.

For a cache-aside design, explicitly define what happens when the cache is unavailable, stale, or overloaded. Falling back to the database without a concurrency limit can cause a sudden database traffic spike.

15. Root Cause #13: Connection Leaks, File Descriptors, and Operating System Limits

Some performance failures originate outside the Java heap.

Applications maintain network connections, sockets, files, threads, and other operating-system resources. A leak or insufficient limit can cause intermittent failures under load.

Examples include:

• HTTP response bodies that are not consumed or closed when required by the client API
• Incorrect lifecycle management for manually created executors
• Repeated creation of HTTP clients instead of reusing appropriately configured clients
• Exhausted file descriptors
• Insufficient container CPU or memory limits
• Operating-system socket exhaustion
• Excessive outbound connections
• Unbounded queues consuming native memory

The correct resource-management approach depends on the API being used. Modern Spring-managed clients and standard try-with-resources patterns simplify lifecycle management, but custom code still requires careful handling.

Useful operating-system and container metrics include:

• Open file descriptors
• Thread count
• Container CPU throttling
• Memory usage and limits
• Network connections and connection states
• Process-level resource utilization

A service can experience severe latency even when Java heap usage looks healthy, especially when container limits or native resources become constrained.

16. Root Cause #14: Missing Backpressure and Overload Protection

A service cannot process unlimited work. When incoming demand exceeds sustainable capacity, it needs a deliberate way to control the amount of work accepted.

Without backpressure or load shedding, the application may continue accepting requests that cannot finish within useful time limits.

Queues grow, memory usage increases, requests expire, and clients retry. Those retries create even more demand.

This can result in a cascading failure.

Example of a cascading failure

Techniques for controlling overload

• Bounded concurrency: Limit simultaneous execution of expensive operations.
• Bounded queues: Prevent pending work from growing without limit.
• Rate limiting: Restrict the request rate per client, tenant, API key, or other relevant identity.
• Bulkheads: Isolate resource pools or concurrency limits for different dependencies.
• Load shedding: Reject work that cannot be processed safely within available capacity.
• Deadlines: Stop waiting when the remaining time is insufficient to complete the operation usefully.
• Graceful degradation: Return a reduced response or use a safe fallback when optional functionality is unavailable.

For example, a recommendation endpoint might return a cached result when its recommendation engine is unhealthy, while a payment endpoint should not silently fabricate a successful payment response.

Choose the overload response based on business semantics. HTTP 429 is suitable for rate limiting, while HTTP 503 may be appropriate when the service is temporarily unable to accept work.

The goal is to preserve system stability rather than maximize the number of requests accepted during an overload event.

17. Root Cause #15: Incorrect Performance Testing

A service that appears fast in development can fail under load because its test environment does not reflect production behavior.

Common testing mistakes include:

• Testing with only a few concurrent users
• Using a small database with unrealistic data
• Running load tests from an overloaded client machine
• Reusing the same cache entries throughout the test
• Ignoring authentication overhead
• Testing only average latency
• Measuring throughput without measuring error rates
• Ignoring warm-up and JIT compilation
• Running all dependencies on unusually fast local infrastructure
• Testing one application instance when production uses a distributed deployment

Understand the load model

There are two common approaches to load generation.

• Closed workload model: A fixed number of virtual users send another request after completing their previous request. As response time increases, the arrival rate may decrease.
• Open workload model: Requests are scheduled to arrive at a target rate independently of previous response completion. If the service slows down, the number of in-flight requests grows.

An open workload model can reveal overload behavior more clearly when the production system receives a relatively independent stream of incoming traffic.

Be aware of coordinated omission: some load-testing methods may under-report latency when the test generator stops issuing requests during periods when the system is slow.

Example using k6

The following test gradually increases virtual-user concurrency:

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '1m', target: 100 },
    { duration: '2m', target: 500 },
    { duration: '2m', target: 1000 },
    { duration: '1m', target: 0 },
  ],
  thresholds: {
    http_req_failed: ['rate<0.01'],
    http_req_duration: ['p(95)<500'],
  },
};

export default function () {
  const response = http.get(
    'http://localhost:8080/api/orders/123'
  );

  check(response, {
    'status is 200': (r) => r.status === 200,
  });

  sleep(1);
}

Save the script as load-test.js and execute it using:

k6 run load-test.js

This is an illustrative starting point, not a definitive 1,000-request capacity test.

The script ramps to 1,000 virtual users, but it does not guarantee 1,000 simultaneous in-flight HTTP requests. Each virtual user waits for its request to complete and then sleeps for one second. The resulting request rate depends on response latency and scheduling.

The 500-millisecond p95 threshold is an example target, not a universal requirement. Adjust it to match your service’s latency objectives.

For a more controlled concurrency test, use a suitable constant-arrival-rate or constant-vus scenario, depending on whether you want to control request arrival rate or virtual-user count. Ensure the load generator itself has enough capacity.

Metrics to capture during a load test

Metric	Why it matters
Requests per second	Measures throughput
p50 latency	Typical response experience
p95 latency	Exposes slower requests
p99 latency	Highlights tail-latency problems
HTTP error rate	Measures reliability
CPU utilization and throttling	Identifies compute constraints
Heap and allocation rate	Identifies memory pressure
GC pause time	Reveals garbage-collection impact
Busy Tomcat threads	Shows request-thread pressure
Hikari active and pending connections	Reveals pool contention
SQL execution time	Identifies database bottlenecks
Downstream latency	Identifies external dependencies
Queue depth and wait time	Reveals accumulated work

Always test with realistic data volumes, request distributions, cache hit rates, and dependency behavior.

18. A Step-by-Step Production Troubleshooting Strategy

When a Spring Boot service becomes slow under 1,000 concurrent requests, avoid changing multiple configuration values simultaneously.

Use a structured investigation.

Step 1: Confirm the symptom

Establish:

• When the latency increase begins
• Which endpoints are affected
• Whether all application instances are affected
• Whether throughput is increasing, stable, or decreasing
• Whether errors rise with latency
• Whether the issue is intermittent or sustained

Compare p50, p95, and p99 latency rather than relying on averages alone.

Step 2: Check CPU and container limits

Inspect process CPU usage and container CPU throttling.

High CPU utilization with a growing runnable-thread count may indicate CPU saturation. Low CPU utilization with very high latency may indicate blocking, queueing, lock contention, or a dependency bottleneck.

Low CPU is not proof of a thread-pool problem, so correlate it with other evidence.

Step 3: Inspect thread dumps

Capture several thread dumps during the slow period.

For example:

jcmd <PID> Thread.print > thread-dump-1.txt

Capture additional dumps a few seconds apart and compare them.

Look for:

• Threads blocked on the same monitor
• Large numbers of threads waiting in JDBC calls
• Threads waiting for HTTP responses
• Repeated stack traces in expensive methods
• Runnable threads competing for CPU
• Excessive worker-thread creation

Thread-state names alone are insufficient to identify the root cause. Inspect stack traces and correlate them with application metrics.

Step 4: Check Tomcat metrics

Determine whether busy request threads approach the configured maximum.

If they do, identify whether the threads are executing application code or waiting for downstream operations.

Increasing the thread count should be a measured experiment, not the first response.

Step 5: Check HikariCP metrics

Inspect active connections, pending requests, acquisition latency, and timeouts.

If the pool is saturated, investigate connection hold times and database behavior before increasing pool size.

Step 6: Inspect SQL execution

Use database monitoring and execution plans to identify slow queries, lock contention, and database saturation.

Check whether query counts increase unexpectedly because of ORM behavior.

Step 7: Inspect downstream services

Measure outbound latency and error rates separately for each dependency.

Check whether HTTP client connection pools are saturated and whether timeouts are correctly configured.

Step 8: Inspect JVM behavior

Use GC logs, JFR, and memory metrics to investigate allocation pressure, long pauses, lock contention, and CPU hotspots.

If memory usage continues to grow after collection, investigate retained objects and potential memory leaks.

Step 9: Apply one targeted fix

Examples include:

• Adding a missing database index
• Removing an N+1 query
• Reducing a transaction’s duration
• Setting a dependency timeout
• Limiting concurrency to a slow downstream service
• Removing unnecessary object allocation
• Introducing bounded queues
• Correcting an overloaded connection pool

Step 10: Repeat the same load test

Compare throughput, p95 and p99 latency, error rate, resource utilization, and dependency health.

A fix is successful only if it improves the relevant outcomes without creating another bottleneck.

19. Spring Boot Actuator and Micrometer Monitoring

Spring Boot Actuator exposes operational endpoints, while Micrometer provides a common instrumentation API for application metrics.

For example, add the Actuator dependency to a Maven project:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

A Prometheus registry can be added when Prometheus is the intended monitoring backend:

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

An example configuration is:

management:
  endpoints:
    web:
      exposure:
        include: health,info,metrics,prometheus

  endpoint:
    health:
      show-details: when_authorized

Do not expose sensitive operational endpoints publicly. Configure authentication, authorization, network restrictions, and appropriate health-detail policies for the deployment environment.

Once metrics are available, a monitoring dashboard can display request latency, error rates, JVM metrics, Tomcat worker utilization, HikariCP pool activity, and downstream call durations.

Example Prometheus alert ideas

Consider alerts for:

• Sustained p95 latency above the service objective
• Increased HTTP 5xx error rate
• Tomcat busy threads approaching their configured limit
• HikariCP pending connections remaining elevated
• Rising connection-acquisition timeouts
• High CPU throttling
• Repeated long GC pauses
• Downstream dependency latency exceeding its budget

Thresholds should reflect baseline behavior and service-level objectives. A short spike in utilization is not always a problem; sustained saturation combined with increased latency or errors is more concerning.

20. How to Design a Spring Boot Service for 1,000 Concurrent Requests

There is no universal configuration that makes every Spring Boot application capable of handling 1,000 concurrent requests. The appropriate design depends on the endpoint’s workload, downstream capacity, latency objective, payload size, and deployment resources.

A practical design should include the following elements.

Application layer

• Use Spring MVC or WebFlux according to the I/O model and application requirements.
• Keep request processing focused and avoid unnecessary synchronous operations.
• Validate input early and limit request payload sizes.
• Paginate large result sets.
• Avoid expensive work on request threads when asynchronous processing is appropriate.

Database layer

• Use a correctly sized HikariCP connection pool.
• Optimize SQL and validate query plans.
• Keep transactions short.
• Avoid N+1 queries.
• Use appropriate indexes.
• Apply caching only where data consistency and invalidation requirements are understood.

Downstream integration

• Configure connection and response timeouts.
• Use circuit breakers and bulkheads where appropriate.
• Apply bounded retries with backoff and jitter.
• Use idempotency controls for retryable state-changing operations.
• Define end-to-end deadlines.

JVM and resource management

• Measure heap requirements and allocation behavior.
• Monitor garbage collection.
• Use thread dumps and JFR when needed.
• Respect container CPU and memory limits.
• Avoid unbounded thread creation and unbounded queues.

Deployment and traffic management

• Use load balancing and horizontal scaling where appropriate.
• Configure health checks and graceful shutdown.
• Apply rate limiting and load shedding.
• Monitor each application instance and shared dependency.
• Validate performance under realistic load.

Horizontal scaling helps when the workload can be distributed across instances and shared dependencies have sufficient capacity. It will not fix a saturated database if every new instance increases the number of competing database requests beyond the database’s capacity.

21. A Practical Capacity Planning Example

Consider a hypothetical Spring Boot order service with these characteristics:

Parameter	Initial value
Tomcat maximum worker threads	200
HikariCP maximum pool size	20
Average database connection hold time	100 ms
Average request duration	250 ms
Target request rate	300 requests/second

The theoretical throughput of database operations is approximately 200 operations per second if every operation holds one connection for 100 milliseconds and the database sustains that workload.

If each incoming request requires two such operations, the rough connection-demand estimate becomes:

300 requests/second × 2 operations/request × 0.1 seconds = 60 connections of average demand.

This exceeds the configured pool size of 20.

The result may be increased connection waiting, higher request latency, and timeouts.

However, this is a simplified model. Actual demand depends on how many operations use connections, whether they execute sequentially or concurrently, how long connections remain checked out, and whether database work can overlap efficiently.

The appropriate response is not necessarily to increase the pool to 60. First, verify database capacity, transaction behavior, and actual connection hold time. Reducing unnecessary database operations or shortening connection occupancy may deliver a larger improvement than increasing concurrency.

Why this matters

A useful capacity plan considers the entire request path:

The sustainable throughput of the system is constrained by the effective capacity of its bottlenecks, not by the largest thread pool in the application.

22. Common Performance Tuning Mistakes

Mistake	Why it can fail
Increasing Tomcat threads immediately	Can move the queue to the database or downstream service
Increasing HikariCP pool size without analysis	Can overload the database
Increasing JVM heap without evidence	May hide symptoms without solving allocation pressure
Switching everything to WebFlux	Blocking dependencies may still consume threads and remain bottlenecks
Adding retries to every failure	Can amplify traffic during an outage
Caching everything	Can create stale-data problems and memory pressure
Scaling instances without database planning	Can multiply database connection demand
Ignoring p99 latency	Can hide severe delays affecting a subset of users
Running tests only on local machines	Can produce unrealistic capacity estimates
Using unbounded executors or queues	Can cause resource exhaustion during overload

The most effective tuning strategy begins with measurements and changes only the resource or behavior responsible for the bottleneck.

23. Frequently Asked Questions (FAQs)

Conclusion

A Spring Boot service becoming slow under 1,000 concurrent requests is not necessarily a sign that Spring Boot cannot handle the workload. More often, it indicates that a resource is saturated, a queue is growing, or a blocking operation is consuming capacity for too long.

The most important performance bottlenecks to investigate are:

1. Tomcat request-thread exhaustion.
2. HikariCP database connection pool saturation.
3. Slow SQL queries and database contention.
4. Blocking external API calls.
5. Poorly configured thread pools.
6. Excessive object allocation and garbage collection.
7. CPU-intensive business logic.
8. Hibernate N+1 queries.
9. Lock contention and shared mutable state.
10. Missing backpressure and overload protection.

The most effective troubleshooting approach is to measure the complete request lifecycle, identify the resource responsible for the delay, implement a targeted fix, and repeat the same load test.

Remember: performance tuning is not about maximizing thread counts. It is about keeping the entire request-processing pipeline within its sustainable capacity while meeting latency and reliability objectives.

A well-designed Spring Boot application combines efficient database access, controlled concurrency, bounded queues, appropriate timeouts, observability, and realistic performance testing. Together, these practices help maintain predictable response times even as traffic increases.

References and further reading

• Spring Boot Reference Documentation: https://docs.spring.io/spring-boot/reference/
• Spring Framework Reference Documentation: https://docs.spring.io/spring-framework/reference/
• Apache Tomcat Configuration: https://tomcat.apache.org/tomcat-10.1-doc/config/
• HikariCP Configuration: https://github.com/brettwooldridge/HikariCP
• Micrometer Documentation: https://docs.micrometer.io/
• Resilience4j Documentation: https://resilience4j.readme.io/
• Java Flight Recorder: https://docs.oracle.com/en/java/javase/21/jfapi/
• PostgreSQL EXPLAIN: https://www.postgresql.org/docs/current/using-explain.html
• k6 Documentation: https://grafana.com/docs/k6/latest/