FM-496 - Upgrade to Spring Boot 4 (#4699)

## Summary

This PR upgrades the API to spring boot 4. An 'non squashed' version of
the PR is available for reference
[here](#4666)

## Upgrade build.gradle for spring boot 4

## Update jdbctemplate usage to reflect null response

## Use updated arguments for ContentCachingFilter

## Update Redis Config Namespace

## Add missing caching request wrapper to filter

## Force nullability on implementation of ConverterFactory

Annotations like @nullable on a Java generic bound do not propagate into
Kotlin’s type system for type parameters. Because the Converter
interface has been updated to use this construct Kotlin can no longer
infer nullabliity

```
public interface Converter<S, T extends @nullable Object> {
	T convert(S source);
}
```

## Use latest health interface

## Update common paging code for new interfaces

## replace SpykBean with MockkSpykBean

This is required for mockk upgrade, see
https://github.com/Ninja-Squad/springmockk/blob/master/README.md#migrating-to-version-5x

## Block first can now return null values

## queryParam is now nullable

## connection pool imports have changed

## Fix exception handling test exceptions

## Fix use of jsonPath.value

## Junit callbacks are no longer nullable

## ApplicationContextInitializer is no longer nullable

## Repository.save can no longer return null

## Fix exception handling unit

## Fix IT data source configuration

## Update spring jackson configuration

This also removes the `serialization` configuration for jackson 3
because it is no longer supported

the `spring.jackson` configuration now applies to jackson3, not
jackson2.

`SerializationFeature.WRITE_DATES_AS_TIMESTAMPS now known as
DateTimeFeature.WRITE_DATES_AS_TIMESTAMPS has been changed to false to
serialize dates as ISO-8601 strings.`

for this reason, we remove the `WRITE_DATES_AS_TIMESTAMPS` config for
jackson 3

Jackson 2 configuration has been added as `spring.jackson2`, matching
the pre spring-boot-4 upgrade configuration.

## Explicitly add and configure webtestclient

## Ensure JPA Attributes are serializable

The JPA specification requires that the entity attributes are
Serializable`

## Remove unused dependencies

## Handle Jackson3 Exception Handling

Before the spring boot 4 upgrade Jackson 2 was used by Spring MVC,
meaning our exception handling responding to unmarshalling issues deal
with Jackson 2 classes.

Spring Boot 4 uses jackson 3 for Spring MVC. This commit updates
exception handling to work solely using Jackson3 for exception handling.

It also simplifies how we use Spring’s `ProblemDetail` type, which is
now correctly placing properties at the top level of the returned JSON
instead of in a ‘properties’ attribute. This is mostly likely due to the
upgrade to Jackson 3. This allows us to simplify the expected problem
JSON in our integration tests, making the JSON match the returned values
we saw from Zalando Problem before moving to Spring Problem

## Method Argument Mismatch now produces a problem

Before upgrading to spring boot 4 `MethodArgumentTypeMismatchException`
would return a generic error. It now returns a JSON Problem, which is
preferred.

## Use Jackson 3 for API `Any` types

As of Spring Boot 4 API marshalling is handled by Jackson 3.

In many places JSON ‘blobs’ are represented in our API model with an
‘Any’ type:

For example, in `ApprovedPremisesApplication` we have

`val document: Any? = null,`

This is populated as follows:

1. Pull JSON from the database as a String
2. Use Jackson to unmarshall this is into an internal Jackson Type
3. Set this value on the `Any` property
4. Spring Web MVC marshalls the `Any` value into JSON in the response

This works fine when the version of Jacksonused in step 2 and 4 are the
same. But if Step 2 uses Jackson2 and Step 4 uses Jackson3 (which is the
configuration we have after upgrading to Spring 4), Jackson3 doesn’t
recognise the Jackson2 native value in the `Any` property and falls back
to default serialization, resulting in something like the following:


`{"array":false,"bigDecimal":false,"bigInteger":false,"binary":false,"boolean":false,"containerNode":true,"double":false,"empty":true,"float":false,"floatingPointNumber":false,"int":false,"integralNumber":false,"long":false,"missingNode":false,"nodeType":"OBJECT","null":false,"number":false,"object":true,"pojo":false,"short":false,"textual":false,"valueNode":false}
`

For this reason we’ve upgraded all places that use Jackson to populate
the `Any` value to use Jackson 3.

Other uses of Jackson will upgraded in subsequent commits

## Migrate some benign jackson2 usage to jackson3

## Pin openapi starter version working version

springdoc/springdoc-openapi#3256 significantly
changed our generated schema, making it incompatible with the typescript
generators and in some places it was incorrect. We're pinning version
3.0.2 until a new version is available reverting this change, as
proposed by springdoc/springdoc-openapi#3276

Author: davidatkinsuk

build.gradle.kts

@@ -1,7 +1,7 @@
 plugins {
-  id("uk.gov.justice.hmpps.gradle-spring-boot") version "9.7.0"
-  kotlin("plugin.spring") version "2.2.21"
-  kotlin("plugin.jpa") version "2.2.21"
+  id("uk.gov.justice.hmpps.gradle-spring-boot") version "10.2.3"
+  kotlin("plugin.spring") version "2.3.20"
+  kotlin("plugin.jpa") version "2.3.20"
   id("dev.detekt") version "2.0.0-alpha.2"
   id("org.owasp.dependencycheck") version "12.2.1"
 }
@@ -21,29 +21,35 @@ configurations.matching { it.name == "detekt" }.all {
 }
 
 dependencies {
-  implementation("uk.gov.justice.service.hmpps:hmpps-kotlin-spring-boot-starter:1.8.2")
+  implementation("uk.gov.justice.service.hmpps:hmpps-kotlin-spring-boot-starter:2.2.0")
+  implementation("org.springframework.boot:spring-boot-starter-flyway")
   implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.10.2")
   implementation("org.springframework.boot:spring-boot-starter-webflux")
   implementation("org.springframework.boot:spring-boot-starter-data-jpa")
   implementation("org.springframework.retry:spring-retry")
-  implementation("io.hypersistence:hypersistence-utils-hibernate-63:3.10.3")
-  // this should match the version of hibernate provided by spring
-  implementation("org.hibernate:hibernate-spatial:6.6.4.Final")
+  implementation("io.hypersistence:hypersistence-utils-hibernate-71:3.15.2")
+  implementation("org.hibernate.orm:hibernate-spatial")
   implementation("org.hibernate.orm:hibernate-jcache")
-  implementation("org.flywaydb:flyway-core")
   implementation("org.springframework.boot:spring-boot-starter-data-redis")
   implementation("org.springframework.boot:spring-boot-starter-cache")
   implementation("com.github.ben-manes.caffeine:caffeine")
   implementation("com.google.guava:guava:33.6.0-jre")
   implementation("org.postgresql:postgresql:42.7.10")
   implementation("org.javers:javers-core:7.11.0")
 
-  implementation("org.springdoc:springdoc-openapi-starter-webmvc-ui:2.8.14")
+  // https://github.com/springdoc/springdoc-openapi/pull/3256 significantly changed our
+  // generated schema, making it incompatible with the typescript generators and in some
+  // places it was incorrect. We're pinning version 3.0.2 until a new version is available
+  // reverting this change, as proposed by https://github.com/springdoc/springdoc-openapi/pull/3276
+  implementation("org.springdoc:springdoc-openapi-starter-webmvc-ui:3.0.2")
+  // this is a transitive dependency of uk.gov.justice.service.hmpps:hmpps-kotlin-spring-boot-autoconfigure
+  // so we need to force a different version
+  implementation("org.springdoc:springdoc-openapi-starter-common:3.0.2")
 
   implementation("org.springframework.boot:spring-boot-starter-security")
   implementation("org.springframework.boot:spring-boot-starter-oauth2-resource-server")
   implementation("org.springframework.boot:spring-boot-starter-oauth2-client")
-  implementation("io.sentry:sentry-spring-boot-starter-jakarta:8.33.0")
+  implementation("io.sentry:sentry-spring-boot-4:8.40.0")
 
   runtimeOnly("org.ehcache:ehcache")
   runtimeOnly("org.flywaydb:flyway-database-postgresql")
@@ -52,10 +58,6 @@ dependencies {
 
   implementation("com.github.doyaaaaaken:kotlin-csv-jvm:1.10.0")
 
-  // This is purely to fix slack notifications as part of OWASP
-  // once we have spring boot 4 we can remove this.
-  implementation("org.webjars:swagger-ui:5.32.2")
-
   implementation("org.jetbrains.kotlinx:dataframe:0.15.0") {
     exclude(group = "org.jetbrains.kotlinx", module = "dataframe-openapi")
   }
@@ -65,20 +67,21 @@ dependencies {
 
   implementation("com.opencsv:opencsv:5.12.0")
 
-  implementation("net.javacrumbs.shedlock:shedlock-spring:6.9.2")
-  implementation("net.javacrumbs.shedlock:shedlock-provider-redis-spring:6.9.2")
+  implementation("net.javacrumbs.shedlock:shedlock-spring:7.7.0")
+  implementation("net.javacrumbs.shedlock:shedlock-provider-redis-spring:7.7.0")
   implementation("org.jetbrains.kotlinx:dataframe-excel:0.15.0")
 
   testImplementation("io.github.bluegroundltd:kfactory:1.0.0")
   testImplementation("io.mockk:mockk:1.14.9")
-  testImplementation("com.github.tomakehurst:wiremock-standalone:3.0.1")
+  testImplementation("org.wiremock.integrations:wiremock-spring-boot:4.2.1")
 
-  testImplementation("uk.gov.justice.service.hmpps:hmpps-kotlin-spring-boot-starter-test:1.8.2")
+  testImplementation("uk.gov.justice.service.hmpps:hmpps-kotlin-spring-boot-starter-test:2.2.0")
 
-  testImplementation("com.ninja-squad:springmockk:4.0.2")
+  testImplementation("com.ninja-squad:springmockk:5.0.1")
+  testImplementation("org.springframework.boot:spring-boot-webtestclient")
   testImplementation("org.zalando:logbook-spring-boot-starter:4.0.3")
 
-  implementation("uk.gov.justice.service.hmpps:hmpps-sqs-spring-boot-starter:5.6.3")
+  implementation("uk.gov.justice.service.hmpps:hmpps-sqs-spring-boot-starter:7.3.1")
 
   implementation("uk.gov.service.notify:notifications-java-client:5.2.1-RELEASE")
 }

src/main/kotlin/uk/gov/justice/digital/hmpps/approvedpremisesapi/cas1/migration/Cas1CapacityPerformanceTestJob.kt

@@ -1,8 +1,8 @@
 package uk.gov.justice.digital.hmpps.approvedpremisesapi.cas1.migration
 
-import com.fasterxml.jackson.databind.json.JsonMapper
 import org.slf4j.LoggerFactory
 import org.springframework.stereotype.Service
+import tools.jackson.databind.json.JsonMapper
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.jpa.entity.ApprovedPremisesRepository
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.migration.MigrationJob
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.service.cas1.Cas1PremisesService

src/main/kotlin/uk/gov/justice/digital/hmpps/approvedpremisesapi/cas1/seed/Cas1RedactAssessmentDetailsSeedJob.kt

@@ -1,10 +1,10 @@
 package uk.gov.justice.digital.hmpps.approvedpremisesapi.cas1.seed
 
-import com.fasterxml.jackson.databind.JsonNode
-import com.fasterxml.jackson.databind.json.JsonMapper
 import org.slf4j.LoggerFactory
 import org.springframework.data.repository.findByIdOrNull
 import org.springframework.stereotype.Component
+import tools.jackson.databind.JsonNode
+import tools.jackson.databind.json.JsonMapper
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.jpa.entity.AssessmentRepository
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.seed.SeedJob
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.service.cas1.Cas1ApplicationTimelineNoteService

src/main/kotlin/uk/gov/justice/digital/hmpps/approvedpremisesapi/cas2/controller/Cas2ApplicationsController.kt

@@ -1,6 +1,5 @@
 package uk.gov.justice.digital.hmpps.approvedpremisesapi.cas2.controller
 
-import com.fasterxml.jackson.databind.json.JsonMapper
 import jakarta.transaction.Transactional
 import org.springframework.http.ResponseEntity
 import org.springframework.web.bind.annotation.GetMapping
@@ -9,6 +8,7 @@ import org.springframework.web.bind.annotation.PostMapping
 import org.springframework.web.bind.annotation.PutMapping
 import org.springframework.web.bind.annotation.RequestBody
 import org.springframework.web.bind.annotation.RequestParam
+import tools.jackson.databind.json.JsonMapper
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.AssignmentType
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.NewApplication
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.SortDirection

src/main/kotlin/uk/gov/justice/digital/hmpps/approvedpremisesapi/cas2/service/Cas2ApplicationService.kt

@@ -1,9 +1,9 @@
 package uk.gov.justice.digital.hmpps.approvedpremisesapi.cas2.service
 
-import com.fasterxml.jackson.databind.json.JsonMapper
 import jakarta.transaction.Transactional
 import org.springframework.beans.factory.annotation.Value
 import org.springframework.stereotype.Service
+import tools.jackson.databind.json.JsonMapper
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.events.cas2.model.Cas2ApplicationSubmittedEvent
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.events.cas2.model.Cas2ApplicationSubmittedEventDetails
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.events.cas2.model.Cas2ApplicationSubmittedEventDetailsSubmittedBy

src/main/kotlin/uk/gov/justice/digital/hmpps/approvedpremisesapi/cas2/transformer/Cas2ApplicationsTransformer.kt

@@ -1,7 +1,7 @@
 package uk.gov.justice.digital.hmpps.approvedpremisesapi.cas2.transformer
 
-import com.fasterxml.jackson.databind.json.JsonMapper
 import org.springframework.stereotype.Component
+import tools.jackson.databind.json.JsonMapper
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.ApplicationOrigin
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.ApplicationStatus
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.ServiceType

src/main/kotlin/uk/gov/justice/digital/hmpps/approvedpremisesapi/cas2/transformer/SubmissionsTransformer.kt

@@ -1,7 +1,7 @@
 package uk.gov.justice.digital.hmpps.approvedpremisesapi.cas2.transformer
 
-import com.fasterxml.jackson.databind.json.JsonMapper
 import org.springframework.stereotype.Component
+import tools.jackson.databind.json.JsonMapper
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.cas2.jpa.entity.Cas2ApplicationEntity
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.cas2.jpa.entity.Cas2ApplicationSummaryEntity
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.cas2.model.Cas2SubmittedApplication

src/main/kotlin/uk/gov/justice/digital/hmpps/approvedpremisesapi/cas2v2/controller/Cas2v2ApplicationController.kt

@@ -1,6 +1,5 @@
 package uk.gov.justice.digital.hmpps.approvedpremisesapi.cas2v2.controller
 
-import com.fasterxml.jackson.databind.json.JsonMapper
 import jakarta.transaction.Transactional
 import org.springframework.http.ResponseEntity
 import org.springframework.web.bind.annotation.GetMapping
@@ -9,6 +8,7 @@ import org.springframework.web.bind.annotation.PostMapping
 import org.springframework.web.bind.annotation.PutMapping
 import org.springframework.web.bind.annotation.RequestBody
 import org.springframework.web.bind.annotation.RequestParam
+import tools.jackson.databind.json.JsonMapper
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.Application
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.ApplicationOrigin
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.NewCas2v2Application

src/main/kotlin/uk/gov/justice/digital/hmpps/approvedpremisesapi/cas2v2/service/Cas2v2ApplicationService.kt

@@ -1,10 +1,10 @@
 package uk.gov.justice.digital.hmpps.approvedpremisesapi.cas2v2.service
 
-import com.fasterxml.jackson.databind.json.JsonMapper
 import jakarta.transaction.Transactional
 import org.springframework.beans.factory.annotation.Value
 import org.springframework.data.jpa.domain.Specification
 import org.springframework.stereotype.Service
+import tools.jackson.databind.json.JsonMapper
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.events.cas2.model.Cas2ApplicationSubmittedEvent
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.events.cas2.model.Cas2ApplicationSubmittedEventDetails
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.events.cas2.model.Cas2ApplicationSubmittedEventDetailsSubmittedBy

src/main/kotlin/uk/gov/justice/digital/hmpps/approvedpremisesapi/cas2v2/transformer/Cas2v2ApplicationsTransformer.kt

@@ -1,7 +1,7 @@
 package uk.gov.justice.digital.hmpps.approvedpremisesapi.cas2v2.transformer
 
-import com.fasterxml.jackson.databind.json.JsonMapper
 import org.springframework.stereotype.Component
+import tools.jackson.databind.json.JsonMapper
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.ApplicationOrigin
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.ApplicationStatus
 import uk.gov.justice.digital.hmpps.approvedpremisesapi.api.model.Cas2v2Application