Skip to content

Unzip zip files in hibernate HBM2DDL_IMPORT_FILES setting #47338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/src/main/asciidoc/hibernate-orm.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -739,6 +739,7 @@ your entity changes or any change to your `import.sql` is immediately picked up
====
By default, in `dev` and `test` modes, Hibernate ORM, upon boot, will read and execute the SQL statements in the `/import.sql` file (if present).
You can change the file name by changing the property `quarkus.hibernate-orm.sql-load-script` in `application.properties`.
You can also provide a `.zip` file in the same way which should contain only the files containing the sql statements to be executed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also update the documentation of the property itself; see:

/**
* Paths to files containing the SQL statements to execute when Hibernate ORM starts.
*
* The files are retrieved from the classpath resources,
* so they must be located in the resources directory (e.g. `src/main/resources`).
*
* The default value for this setting differs depending on the Quarkus launch mode:
*
* * In dev and test modes, it defaults to `import.sql`.
* Simply add an `import.sql` file in the root of your resources directory
* and it will be picked up without having to set this property.
* Pass `no-file` to force Hibernate ORM to ignore the SQL import file.
* * In production mode, it defaults to `no-file`.
* It means Hibernate ORM won't try to execute any SQL import file by default.
* Pass an explicit value to force Hibernate ORM to execute the SQL import file.
*
* If you need different SQL statements between dev mode, test (`@QuarkusTest`) and in production, use Quarkus
* https://quarkus.io/guides/config#configuration-profiles[configuration profiles facility].
*
* [source,property]
* .application.properties
* ----
* %dev.quarkus.hibernate-orm.sql-load-script = import-dev.sql
* %test.quarkus.hibernate-orm.sql-load-script = import-test.sql
* %prod.quarkus.hibernate-orm.sql-load-script = no-file
* ----
*
* [NOTE]
* ====
* Quarkus supports `.sql` file with SQL statements or comments spread over multiple lines.
* Each SQL statement must be terminated by a semicolon.
* ====
*
* @asciidoclet
*/
// @formatter:on
@ConfigDocDefault("import.sql in dev and test modes ; no-file otherwise")
Optional<List<@WithConverter(TrimmedStringConverter.class) String>> sqlLoadScript();

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

====

The second approach is to use `quarkus.hibernate-orm.database.generation=update`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,8 @@ public interface HibernateOrmConfigPersistenceUnit {
*
* [NOTE]
* ====
* Quarkus supports `.sql` file with SQL statements or comments spread over multiple lines.
* Quarkus supports files with SQL statements or comments spread over multiple lines,
* or `.zip` files containing those files.
* Each SQL statement must be terminated by a semicolon.
* ====
*
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package io.quarkus.hibernate.orm.sql_load_script;

import org.hamcrest.Matchers;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import io.quarkus.hibernate.orm.MyEntity;
import io.quarkus.test.QuarkusUnitTest;
import io.restassured.RestAssured;

public class ImportMultipleSqlLoadScriptsAsZipFileTestCase {
@RegisterExtension
static QuarkusUnitTest runner = new QuarkusUnitTest()
.withApplicationRoot((jar) -> jar
.addClasses(MyEntity.class, SqlLoadScriptTestResource.class)
.addAsResource("application-multiple-load-script-files-as-zip-file-test.properties",
"application.properties")
.addAsResource("multiple-load-script-files.zip"));

@Test
public void testMultipleLoadScriptFilesAsZipFile() {
String name1 = "import-1.sql load script entity";
String name2 = "import-2.sql load script entity";

RestAssured.when().get("/orm-sql-load-script/1").then().body(Matchers.is(name1));
RestAssured.when().get("/orm-sql-load-script/2").then().body(Matchers.is(name2));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
package io.quarkus.hibernate.orm.sql_load_script;

import org.hamcrest.Matchers;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import io.quarkus.hibernate.orm.MyEntity;
import io.quarkus.test.QuarkusUnitTest;
import io.restassured.RestAssured;

public class ImportSqlLoadScriptAsZipFileTestCase {
@RegisterExtension
static QuarkusUnitTest runner = new QuarkusUnitTest()
.withApplicationRoot((jar) -> jar
.addClasses(MyEntity.class, SqlLoadScriptTestResource.class)
.addAsResource("application-load-script-as-zip-file-test.properties", "application.properties")
.addAsResource("load-script-test.zip"));

@Test
public void testSqlLoadScriptAsZipFile() {
String name = "other-load-script sql load script entity";
RestAssured.when().get("/orm-sql-load-script/3").then().body(Matchers.is(name));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
package io.quarkus.hibernate.orm.sql_load_script;

import org.hamcrest.Matchers;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import io.quarkus.hibernate.orm.MyEntity;
import io.quarkus.test.QuarkusUnitTest;
import io.restassured.RestAssured;

public class ImportSqlLoadScriptsAsMultipleZipFilesTestCase {
@RegisterExtension
static QuarkusUnitTest runner = new QuarkusUnitTest()
.withApplicationRoot((jar) -> jar
.addClasses(MyEntity.class, SqlLoadScriptTestResource.class)
.addAsResource("application-load-scripts-as-multiple-zip-files-test.properties",
"application.properties")
.addAsResource("import-multiple-load-scripts-1.zip")
.addAsResource("import-multiple-load-scripts-2.zip"));

@Test
public void testSqlLoadScriptsAsMultipleZipFiles() {
String name1 = "import-1.sql load script entity";
String name2 = "import-2.sql load script entity";

RestAssured.when().get("/orm-sql-load-script/1").then().body(Matchers.is(name1));
RestAssured.when().get("/orm-sql-load-script/2").then().body(Matchers.is(name2));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package io.quarkus.hibernate.orm.sql_load_script;

import org.hamcrest.Matchers;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import io.quarkus.hibernate.orm.MyEntity;
import io.quarkus.test.QuarkusUnitTest;
import io.restassured.RestAssured;

public class ImportSqlLoadScriptsAsZipFilesAndSqlFileTestCase {
@RegisterExtension
static QuarkusUnitTest runner = new QuarkusUnitTest()
.withApplicationRoot((jar) -> jar
.addClasses(MyEntity.class, SqlLoadScriptTestResource.class)
.addAsResource("application-load-scripts-as-multiple-zip-files-and-sql-file-test.properties",
"application.properties")
.addAsResource("load-script-test.sql")
.addAsResource("import-multiple-load-scripts-1.zip")
.addAsResource("import-multiple-load-scripts-2.zip"));

@Test
public void testSqlLoadScriptsAsZipFilesAndSqlFile() {
String name = "other-load-script sql load script entity";
String name1 = "import-1.sql load script entity";
String name2 = "import-2.sql load script entity";

RestAssured.when().get("/orm-sql-load-script/1").then().body(Matchers.is(name1));
RestAssured.when().get("/orm-sql-load-script/2").then().body(Matchers.is(name2));
RestAssured.when().get("/orm-sql-load-script/3").then().body(Matchers.is(name));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
quarkus.datasource.db-kind=h2
quarkus.datasource.jdbc.url=jdbc:h2:mem:test

quarkus.hibernate-orm.database.generation=drop-and-create
quarkus.hibernate-orm.sql-load-script=load-script-test.zip
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
quarkus.datasource.db-kind=h2
quarkus.datasource.jdbc.url=jdbc:h2:mem:test

quarkus.hibernate-orm.database.generation=drop-and-create
quarkus.hibernate-orm.sql-load-script=load-script-test.sql, import-multiple-load-scripts-1.zip, import-multiple-load-scripts-2.zip
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
quarkus.datasource.db-kind=h2
quarkus.datasource.jdbc.url=jdbc:h2:mem:test

quarkus.hibernate-orm.database.generation=drop-and-create
quarkus.hibernate-orm.sql-load-script=import-multiple-load-scripts-1.zip, import-multiple-load-scripts-2.zip
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
quarkus.datasource.db-kind=h2
quarkus.datasource.jdbc.url=jdbc:h2:mem:test

quarkus.hibernate-orm.database.generation=drop-and-create
quarkus.hibernate-orm.sql-load-script=multiple-load-script-files.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package io.quarkus.hibernate.orm.runtime;

public final class Constants {
public static final String COMMA = ",";
public static final String ZIP_FILE_EXTENSION = ".zip";
public static final String SQL_LOAD_SCRIPT_UNZIPPED_DIR_PREFIX = "import-sql-unzip-";

private Constants() {
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@ private RuntimeSettings buildRuntimeSettings(String persistenceUnitName, Recorde
final BuildTimeSettings buildTimeSettings = recordedState.getBuildTimeSettings();
final IntegrationSettings integrationSettings = recordedState.getIntegrationSettings();
Builder runtimeSettingsBuilder = new Builder(buildTimeSettings, integrationSettings);
unzipZipFilesAndReplaceZipsInImportFiles(runtimeSettingsBuilder);

Optional<String> dataSourceName = recordedState.getBuildTimeSettings().getSource().getDataSource();
if (dataSourceName.isPresent()) {
Expand Down Expand Up @@ -288,6 +289,13 @@ private RuntimeSettings buildRuntimeSettings(String persistenceUnitName, Recorde
return runtimeSettingsBuilder.build();
}

private void unzipZipFilesAndReplaceZipsInImportFiles(Builder runtimeSettingsBuilder) {
String newValue = SchemaToolingUtil.unzipZipFilesAndReplaceZips(
(String) runtimeSettingsBuilder.get(AvailableSettings.HBM2DDL_IMPORT_FILES));
runtimeSettingsBuilder.put(AvailableSettings.HBM2DDL_IMPORT_FILES, null);
runtimeSettingsBuilder.put(AvailableSettings.JAKARTA_HBM2DDL_LOAD_SCRIPT_SOURCE, newValue);
}

private StandardServiceRegistry rewireMetadataAndExtractServiceRegistry(String persistenceUnitName, RecordedState rs,
HibernateOrmRuntimeConfigPersistenceUnit puConfig, RuntimeSettings runtimeSettings) {
PreconfiguredServiceRegistryBuilder serviceRegistryBuilder = new PreconfiguredServiceRegistryBuilder(
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
package io.quarkus.hibernate.orm.runtime;

import static io.quarkus.hibernate.orm.runtime.Constants.COMMA;
import static io.quarkus.hibernate.orm.runtime.Constants.SQL_LOAD_SCRIPT_UNZIPPED_DIR_PREFIX;
import static io.quarkus.hibernate.orm.runtime.Constants.ZIP_FILE_EXTENSION;

import java.net.URL;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.LinkedList;
import java.util.List;

import org.jboss.logging.Logger;

import io.quarkus.fs.util.ZipUtils;

public class SchemaToolingUtil {
private static final Logger log = Logger.getLogger(SchemaToolingUtil.class);

public static String unzipZipFilesAndReplaceZips(String commaSeparatedFileNames) {
List<String> unzippedFilesNames = new LinkedList<>();
if (commaSeparatedFileNames != null) {
String[] fileNames = commaSeparatedFileNames.split(COMMA);
for (String fileName : fileNames) {
if (fileName.endsWith(ZIP_FILE_EXTENSION)) {
try {
Path unzipDir = Files.createTempDirectory(SQL_LOAD_SCRIPT_UNZIPPED_DIR_PREFIX);
URL resource = Thread.currentThread()
.getContextClassLoader()
.getResource(fileName);
Path zipFile = Paths.get(resource.toURI());
ZipUtils.unzip(zipFile, unzipDir);
try (DirectoryStream<Path> paths = Files.newDirectoryStream(unzipDir)) {
for (Path path : paths) {
unzippedFilesNames.add(path.toAbsolutePath().toString());
}
}
} catch (Exception e) {
log.errorf("Error unzipping import file %s: %s", fileName, e.getMessage());
throw new IllegalStateException(e);
}
} else {
unzippedFilesNames.add(fileName);
}
}
return String.join(COMMA, unzippedFilesNames);
} else {
return null;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
import io.quarkus.hibernate.orm.runtime.PersistenceUnitsHolder;
import io.quarkus.hibernate.orm.runtime.RuntimeSettings;
import io.quarkus.hibernate.orm.runtime.RuntimeSettings.Builder;
import io.quarkus.hibernate.orm.runtime.SchemaToolingUtil;
import io.quarkus.hibernate.orm.runtime.boot.QuarkusPersistenceUnitDescriptor;
import io.quarkus.hibernate.orm.runtime.integration.HibernateOrmIntegrationRuntimeDescriptor;
import io.quarkus.hibernate.orm.runtime.integration.HibernateOrmIntegrationRuntimeInitListener;
Expand Down Expand Up @@ -158,6 +159,7 @@ private EntityManagerFactoryBuilder getEntityManagerFactoryBuilderOrNull(String
final IntegrationSettings integrationSettings = recordedState.getIntegrationSettings();
RuntimeSettings.Builder runtimeSettingsBuilder = new RuntimeSettings.Builder(buildTimeSettings,
integrationSettings);
unzipZipFilesAndReplaceZipsInImportFiles(runtimeSettingsBuilder);

var puConfig = hibernateOrmRuntimeConfig.persistenceUnits().get(persistenceUnit.getConfigurationName());
if (puConfig.active().isPresent() && !puConfig.active().get()) {
Expand Down Expand Up @@ -224,6 +226,13 @@ private EntityManagerFactoryBuilder getEntityManagerFactoryBuilderOrNull(String
return null;
}

private void unzipZipFilesAndReplaceZipsInImportFiles(Builder runtimeSettingsBuilder) {
String newValue = SchemaToolingUtil.unzipZipFilesAndReplaceZips(
(String) runtimeSettingsBuilder.get(AvailableSettings.HBM2DDL_IMPORT_FILES));
runtimeSettingsBuilder.put(AvailableSettings.HBM2DDL_IMPORT_FILES, null);
runtimeSettingsBuilder.put(AvailableSettings.JAKARTA_HBM2DDL_LOAD_SCRIPT_SOURCE, newValue);
}

private StandardServiceRegistry rewireMetadataAndExtractServiceRegistry(String persistenceUnitName, RecordedState rs,
RuntimeSettings runtimeSettings, HibernateOrmRuntimeConfigPersistenceUnit puConfig) {
PreconfiguredReactiveServiceRegistryBuilder serviceRegistryBuilder = new PreconfiguredReactiveServiceRegistryBuilder(
Expand Down
Loading