Spring boot jpa querydsl 多数据源使用入坑

背景

  • 随着同类项目越来越多，从产品、架构角度考虑，需要把重复的、基础的东西抽离出来，用以复用
  • 用户并没有达到指数级增长，从部署的角度考虑，暂时不需要做微服务
  • 因此，目前只做数据库、工程的拆分，在部署层面仍然使用单一服务形式。
  • 在以上形式下，就需要配置多数据源，下面是入坑指南

入坑过程

项目单数据源初始状态

项目的基础包

  • spring boot
  • spring data jpa
  • querydsl
  • mysql8.0
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- AuditingEntityListener -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<!-- mysql -->
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
</dependency>
<!--queryDSL-->
<dependency>
    <groupId>com.querydsl</groupId>
    <artifactId>querydsl-jpa</artifactId>
    <version>${querydsl.version}</version>
</dependency>
<dependency>
    <groupId>com.querydsl</groupId>
    <artifactId>querydsl-apt</artifactId>
    <version>${querydsl.version}</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <optional>true</optional>
</dependency>

初步的想法和说明

  • 项目module怎么搭根据你们自己的要求搭建，但是请注意Spring习惯使用packageScan，为了后续搭建方便，最好对多模块进行分包
  • 对多数据源的配置和使用，一条基本原则就是多数据源配置尽可能的与业务开发无关，最好开发无感，需要事务的时候和原来一样使用@Transactional
  • 原先想要拆分的Entity之间通过@ManyToOne、@OneToMany等JPA的标准接口做的关联关系，我们希望：
    • 只做不同数据源的JPA的packageScan，让对应的JPA自动建立到对应的库中
    • 底层可以支持自动识别并进行跨库查询，当我使用由关联关系的Entity时
  • 如果底层支持第二条，我们再考虑事务和Session

配置多数据源

简单的数据源配置

如何配置多数据源

Configure Two DataSources

If you need to configure multiple data sources, you can apply the same tricks that are described in the previous section. You must, however, mark one of the DataSource instances as @Primary, because various auto-configurations down the road expect to be able to get one by type.
If you create your own DataSource, the auto-configuration backs off. In the following example, we provide the exact same feature set as the auto-configuration provides on the primary data source:

@Bean
@Primary
@ConfigurationProperties("app.datasource.first")
public DataSourceProperties firstDataSourceProperties() {
    return new DataSourceProperties();
}

@Bean
@Primary
@ConfigurationProperties("app.datasource.first.configuration")
public HikariDataSource firstDataSource() {
    return firstDataSourceProperties().initializeDataSourceBuilder().type(HikariDataSource.class).build();
}

// u can build the second datasource as same as shown above
@Bean
@ConfigurationProperties("app.datasource.second")
public BasicDataSource secondDataSource() {
    return DataSourceBuilder.create().type(BasicDataSource.class).build();
}
  • 当你使用多数据源配置datasources时，注意设置@Primary
  • 推荐使用DataSourceProperties方式，type推荐HikariDataSource，这个是Spring单数据源的默认类型

Use Two EntityManagers

Even if the default EntityManagerFactory works fine, you need to define a new one, otherwise the presence of the second bean of that type switches off the default. You can use the EntityManagerBuilder provided by Spring Boot to help you to create one. Alternatively, you can use the LocalContainerEntityManagerFactoryBean directly from Spring ORM, as shown in the following example:

@Bean
public LocalContainerEntityManagerFactoryBean customerEntityManagerFactory(EntityManagerFactoryBuilder builder) {
    return builder
            .dataSource(customerDataSource())
            .packages(Customer.class)
            .persistenceUnit("customers")
            .build();
}

@Bean
public LocalContainerEntityManagerFactoryBean orderEntityManagerFactory(EntityManagerFactoryBuilder builder) {
    return builder
            .dataSource(orderDataSource())
            .packages(Order.class)
            .persistenceUnit("orders")
            .build();
}
  • 一旦你开启多数据源配置，自定义EntityManagerFactory无法避免
  • 你有两种方式创建它，EntityManagerBuilder或者直接new LocalContainerEntityManagerFactoryBean
  • 注意！！！！！！！！！！！！！！下面的说明如果你没有看将会有很多坑，后面会一点一点补完。大致的意思就是说最好用Spring自己的EntityManagerFactoryBuilder去生成LocalContainerEntityManagerFactoryBean，否则后果自负

When you create a bean for LocalContainerEntityManagerFactoryBean yourself, any customization that was applied during the creation of the auto-configured LocalContainerEntityManagerFactoryBean is lost. For example, in case of Hibernate, any properties under the spring.jpa.hibernate prefix will not be automatically applied to your LocalContainerEntityManagerFactoryBean. If you were relying on these properties for configuring things like the naming strategy or the DDL mode, you will need to explicitly configure that when creating the LocalContainerEntityManagerFactoryBean bean. On the other hand, properties that get applied to the auto-configured EntityManagerFactoryBuilder, which are specified via spring.jpa.properties, will automatically be applied, provided you use the auto-configured EntityManagerFactoryBuilder to build the LocalContainerEntityManagerFactoryBean bean.

  • 注：这里的代码用的bean name和上面db config bean没有关系，只是个例子，读者自行修改以适应自己的项目，下同，本质上就是配置两个数据源，最后会给出完整配置和源代码github地址，先了解大概即可

源码分析

EntityManagerFactoryBuilder
  • 位置：spring-boot-autoconfigure-2.1.4.RELEASE.jar
  • JpaBaseConfiguration
    • 注意@EnableConfigurationProperties(JpaProperties.class)，实例化属性取决于JpaProperties.class配置的spring.jpa，稍后看ymal文件即可
    • @ConditionalOnMissingBean表明如果我们自己不实例化EntityManagerFactoryBuilder，则Spring会帮我们构造一个
package org.springframework.boot.autoconfigure.orm.jpa;
@Configuration
@EnableConfigurationProperties(JpaProperties.class)
@Import(DataSourceInitializedPublisher.Registrar.class)
public abstract class JpaBaseConfiguration implements BeanFactoryAware {

    private final DataSource dataSource;

    private final JpaProperties properties;

    private final JtaTransactionManager jtaTransactionManager;

    private final TransactionManagerCustomizers transactionManagerCustomizers;

    private ConfigurableListableBeanFactory beanFactory;

    protected JpaBaseConfiguration(DataSource dataSource, JpaProperties properties,
            ObjectProvider<JtaTransactionManager> jtaTransactionManager,
            ObjectProvider<TransactionManagerCustomizers> transactionManagerCustomizers) {
        this.dataSource = dataSource;
        this.properties = properties;
        this.jtaTransactionManager = jtaTransactionManager.getIfAvailable();
        this.transactionManagerCustomizers = transactionManagerCustomizers
                .getIfAvailable();
    }

    @Bean
    @ConditionalOnMissingBean
    public PlatformTransactionManager transactionManager() {
        JpaTransactionManager transactionManager = new JpaTransactionManager();
        if (this.transactionManagerCustomizers != null) {
            this.transactionManagerCustomizers.customize(transactionManager);
        }
        return transactionManager;
    }

    @Bean
    @ConditionalOnMissingBean
    public JpaVendorAdapter jpaVendorAdapter() {
        AbstractJpaVendorAdapter adapter = createJpaVendorAdapter();
        adapter.setShowSql(this.properties.isShowSql());
        adapter.setDatabase(this.properties.determineDatabase(this.dataSource));
        adapter.setDatabasePlatform(this.properties.getDatabasePlatform());
        adapter.setGenerateDdl(this.properties.isGenerateDdl());
        return adapter;
    }

    @Bean
    @ConditionalOnMissingBean
    public EntityManagerFactoryBuilder entityManagerFactoryBuilder(
            JpaVendorAdapter jpaVendorAdapter,
            ObjectProvider<PersistenceUnitManager> persistenceUnitManager,
            ObjectProvider<EntityManagerFactoryBuilderCustomizer> customizers) {
        EntityManagerFactoryBuilder builder = new EntityManagerFactoryBuilder(
                jpaVendorAdapter, this.properties.getProperties(),
                persistenceUnitManager.getIfAvailable());
        customizers.orderedStream()
                .forEach((customizer) -> customizer.customize(builder));
        return builder;
    }

    @Bean
    @Primary
    @ConditionalOnMissingBean({ LocalContainerEntityManagerFactoryBean.class,
            EntityManagerFactory.class })
    public LocalContainerEntityManagerFactoryBean entityManagerFactory(
            EntityManagerFactoryBuilder factoryBuilder) {
        Map<String, Object> vendorProperties = getVendorProperties();
        customizeVendorProperties(vendorProperties);
        return factoryBuilder.dataSource(this.dataSource).packages(getPackagesToScan())
                .properties(vendorProperties).mappingResources(getMappingResources())
                .jta(isJta()).build();
    }
}
HibernateJpaConfiguration
  • JpaBaseConfiguration的默认实现
  • 注意@ConditionalOnSingleCandidate(DataSource.class)，因为我们有多个DataSource实例，所以它会失效，也就是说hibernate的默认配置无法生效
  • @EnableConfigurationProperties(HibernateProperties.class)，表明spring.jpa.hibernate也无效
@Configuration
@EnableConfigurationProperties(HibernateProperties.class)
@ConditionalOnSingleCandidate(DataSource.class)
class HibernateJpaConfiguration extends JpaBaseConfiguration {...}
自动装配
  • org.springframework.boot.autoconfigure.orm.jpa.JpaRepositoriesAutoConfiguration
  • org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration
  • org.springframework.boot.autoconfigure.data.jpa.JpaRepositoriesAutoConfigureRegistrar

PlatformTransactionManager

The configuration above almost works on its own. To complete the picture, you need to configure TransactionManagers for the two EntityManagers as well. If you mark one of them as @Primary, it could be picked up by the default JpaTransactionManager in Spring Boot. The other would have to be explicitly injected into a new instance. Alternatively, you might be able to use a JTA transaction manager that spans both.

@Bean(name = Constant.TRANSACTION_MANAGER_PROJECT)
@Primary
public JpaTransactionManager transactionManager() {
    JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
    jpaTransactionManager.setEntityManagerFactory(entityManagerFactoryBean().getObject());
    return jpaTransactionManager;
}

@Bean(name = Constant.TRANSACTION_MANAGER_EVENT)
public JpaTransactionManager transactionManager() {
    JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
    jpaTransactionManager.setEntityManagerFactory(entityManagerFactoryBean().getObject());
    return jpaTransactionManager;
}
  • spring没有给demo，这是我自己加的，差不多就是这个样子

EntityManager & JPAQueryFactory

@Bean(name = Constant.ENTITY_MANAGER_PROJECT)
@Primary
public EntityManager entityManager() {
    return entityManagerFactoryBean().getObject().createEntityManager();
}

@Bean(name = Constant.JPA_QUERY_FACTORY_PROJECT)
@Primary
public JPAQueryFactory jpaQueryFactory() {
    return new JPAQueryFactory(entityManager());
}

@Bean(name = Constant.ENTITY_MANAGER_EVENT)
public EntityManager entityManager() {
    return entityManagerFactoryBean().getObject().createEntityManager();
}

@Bean(name = Constant.JPA_QUERY_FACTORY_EVENT)
public JPAQueryFactory jpaQueryFactory() {
    return new JPAQueryFactory(entityManager());
}
  • 因为我们项目用到了querydsl，所以需要JPAQueryFactory

@EnableJpaRepositories

If you use Spring Data, you need to configure @EnableJpaRepositories accordingly, as shown in the following example

@Configuration
@EnableJpaRepositories(
        basePackageClasses = {ProjectRepository.class},
        entityManagerFactoryRef = Constant.ENTITY_MANAGER_FACTORY_PROJECT,
        transactionManagerRef = Constant.TRANSACTION_MANAGER_PROJECT)
public class JpaConfigurationProject {
    ...
}

@Configuration
@EnableJpaRepositories(
        basePackageClasses = {EventRepository.class},
        entityManagerFactoryRef = Constant.ENTITY_MANAGER_FACTORY_EVENT,
        transactionManagerRef = Constant.TRANSACTION_MANAGER_EVENT)
public class JpaConfigurationEvent {
    ...
}
  • 你也可以使用basePackages配置扫描

Initialize a Database Using JPA

JPA has features for DDL generation, and these can be set up to run on startup against the database. This is controlled through two external properties:

  1. spring.jpa.generate-ddl (boolean) switches the feature on and off and is vendor independent.
  2. spring.jpa.hibernate.ddl-auto (enum) is a Hibernate feature that controls the behavior in a more fine-grained way. This feature is described in more detail later in this guide.

两种方式都可以，第一种更通用，第二种具体到hibernate，粒度更细

跨服务、数据库应该通过服务接口实现，而不是DB

移除@OneToOne、@ManyToOne

  • 实体关系依赖改为ID依赖

默认hibernate配置失效

See HibernateJpaAutoConfiguration and JpaBaseConfiguration for more details.

方式1（Spring推荐）

  • Hibernate uses two different naming strategies to map names from the object model to the corresponding database names. The fully qualified class name of the physical and the implicit strategy implementations can be configured by setting the spring.jpa.hibernate.naming.physical-strategy and spring.jpa.hibernate.naming.implicit-strategy properties, respectively. Alternatively, if ImplicitNamingStrategy or PhysicalNamingStrategy beans are available in the application context, Hibernate will be automatically configured to use them.
  • By default, Spring Boot configures the physical naming strategy with SpringPhysicalNamingStrategy. This implementation provides the same table structure as Hibernate 4: all dots are replaced by underscores and camel casing is replaced by underscores as well. Additionally, by default, all table names are generated in lower case. For example, a TelephoneNumber entity is mapped to the telephone_number table. If your schema requires mixed-case identifiers, define a custom SpringPhysicalNamingStrategy bean, as shown in the following example:
@Bean
SpringPhysicalNamingStrategy caseSensitivePhysicalNamingStrategy() {
    return new SpringPhysicalNamingStrategy() {

        @Override
        protected boolean isCaseInsensitive(JdbcEnvironment jdbcEnvironment) {
            return false;
        }

    };
}

@Bean
public PhysicalNamingStrategy physicalNamingStrategy() {
    return new PhysicalNamingStrategyStandardImpl();
}

方式2（Spring推荐）

spring.jpa.hibernate.naming.physical-strategy=org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl

方式3（自定义HibernateConfiguration）

  • 上面源码时也说过，HibernateJpaConfiguration只对单数据源生效，我们直接模仿它写一个实现类给LocalContainerEntityManagerFactoryBean配置即可
package cc.gegee.pangolin.config.atomikos;

import lombok.val;
import org.hibernate.boot.model.naming.ImplicitNamingStrategy;
import org.hibernate.boot.model.naming.PhysicalNamingStrategy;
import org.hibernate.cfg.AvailableSettings;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
import org.springframework.boot.autoconfigure.orm.jpa.HibernateProperties;
import org.springframework.boot.autoconfigure.orm.jpa.HibernatePropertiesCustomizer;
import org.springframework.boot.autoconfigure.orm.jpa.HibernateSettings;
import org.springframework.boot.autoconfigure.orm.jpa.JpaProperties;
import org.springframework.boot.jdbc.EmbeddedDatabaseConnection;
import org.springframework.boot.jdbc.SchemaManagement;
import org.springframework.boot.jdbc.SchemaManagementProvider;
import org.springframework.context.annotation.Configuration;
import org.springframework.orm.hibernate5.SpringBeanContainer;
import org.springframework.util.ClassUtils;

import javax.sql.DataSource;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import java.util.stream.StreamSupport;

/**
 * copy from org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration
 * used for default hibernate setting
 */
@Configuration
public class HibernateConfiguration {

    private final ObjectProvider<SchemaManagementProvider> providers;

    private final HibernateProperties hibernateProperties;

    private final JpaProperties properties;

    private final ObjectProvider<PhysicalNamingStrategy> physicalNamingStrategy;

    private final ObjectProvider<ImplicitNamingStrategy> implicitNamingStrategy;

    private final ConfigurableListableBeanFactory beanFactory;

    private final ObjectProvider<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers;

    public HibernateConfiguration(ObjectProvider<SchemaManagementProvider> providers, HibernateProperties hibernateProperties, JpaProperties properties, ObjectProvider<PhysicalNamingStrategy> physicalNamingStrategy, ObjectProvider<ImplicitNamingStrategy> implicitNamingStrategy, ConfigurableListableBeanFactory beanFactory, ObjectProvider<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers) {
        this.providers = providers;
        this.hibernateProperties = hibernateProperties;
        this.properties = properties;
        this.physicalNamingStrategy = physicalNamingStrategy;
        this.implicitNamingStrategy = implicitNamingStrategy;
        this.beanFactory = beanFactory;
        this.hibernatePropertiesCustomizers = hibernatePropertiesCustomizers;
    }

    /**
     * 获取配置文件信息
     */
    public Map<String, Object> getVendorProperties(DataSource dataSource) {
        List<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers = determineHibernatePropertiesCustomizers(
                physicalNamingStrategy.getIfAvailable(),
                implicitNamingStrategy.getIfAvailable(), beanFactory,
                this.hibernatePropertiesCustomizers.orderedStream()
                        .collect(Collectors.toList()));
        Supplier<String> defaultDdlMode = () -> new HibernateDefaultDdlAutoProvider(providers)
                .getDefaultDdlAuto(dataSource);
        val vendorProperties = new LinkedHashMap<>(this.hibernateProperties.determineHibernateProperties(
                properties.getProperties(),
                new HibernateSettings().ddlAuto(defaultDdlMode)
                        .hibernatePropertiesCustomizers(
                                hibernatePropertiesCustomizers)));
        // DdlTransactionIsolatorJtaImpl could not locate TransactionManager to suspend any current transaction; base JtaPlatform impl
        vendorProperties.put("hibernate.transaction.jta.platform", AtomikosJtaPlatform.class.getName());
        // i can not find any reason to set this value, if any question happen, try to set it
        vendorProperties.put("javax.persistence.transactionType", "JTA");
//        vendorProperties.put("javax.persistence.cache.storeMode", CacheStoreMode.BYPASS);
        return vendorProperties;
    }

    /**
     * 命名策略自动判断
     */
    private List<HibernatePropertiesCustomizer> determineHibernatePropertiesCustomizers(
            PhysicalNamingStrategy physicalNamingStrategy,
            ImplicitNamingStrategy implicitNamingStrategy,
            ConfigurableListableBeanFactory beanFactory,
            List<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers) {
        List<HibernatePropertiesCustomizer> customizers = new ArrayList<>();
        if (ClassUtils.isPresent(
                "org.hibernate.resource.beans.container.spi.BeanContainer",
                getClass().getClassLoader())) {
            customizers
                    .add((properties) -> properties.put(AvailableSettings.BEAN_CONTAINER,
                            new SpringBeanContainer(beanFactory)));
        }
        if (physicalNamingStrategy != null || implicitNamingStrategy != null) {
            customizers.add(new NamingStrategiesHibernatePropertiesCustomizer(
                    physicalNamingStrategy, implicitNamingStrategy));
        }
        customizers.addAll(hibernatePropertiesCustomizers);
        return customizers;
    }

    /**
     * 自动进行建表操作
     */
    static class HibernateDefaultDdlAutoProvider implements SchemaManagementProvider {

        private final Iterable<SchemaManagementProvider> providers;

        HibernateDefaultDdlAutoProvider(Iterable<SchemaManagementProvider> providers) {
            this.providers = providers;
        }

        public String getDefaultDdlAuto(DataSource dataSource) {
            if (!EmbeddedDatabaseConnection.isEmbedded(dataSource)) {
                return "none";
            }
            SchemaManagement schemaManagement = getSchemaManagement(dataSource);
            if (SchemaManagement.MANAGED.equals(schemaManagement)) {
                return "none";
            }
            return "create-drop";

        }

        @Override
        public SchemaManagement getSchemaManagement(DataSource dataSource) {
            return StreamSupport.stream(this.providers.spliterator(), false)
                    .map((provider) -> provider.getSchemaManagement(dataSource))
                    .filter(SchemaManagement.MANAGED::equals).findFirst()
                    .orElse(SchemaManagement.UNMANAGED);
        }

    }

    private static class NamingStrategiesHibernatePropertiesCustomizer
            implements HibernatePropertiesCustomizer {

        private final PhysicalNamingStrategy physicalNamingStrategy;

        private final ImplicitNamingStrategy implicitNamingStrategy;

        NamingStrategiesHibernatePropertiesCustomizer(PhysicalNamingStrategy physicalNamingStrategy, ImplicitNamingStrategy implicitNamingStrategy) {
            this.physicalNamingStrategy = physicalNamingStrategy;
            this.implicitNamingStrategy = implicitNamingStrategy;
        }

        /**
         * 数据库命名映射策略
         *
         * @param hibernateProperties the JPA vendor properties to customize
         */
        @Override
        public void customize(Map<String, Object> hibernateProperties) {
            if (this.physicalNamingStrategy != null) {
                hibernateProperties.put("hibernate.physical_naming_strategy", this.physicalNamingStrategy);
            }
            if (this.implicitNamingStrategy != null) {
                hibernateProperties.put("hibernate.implicit_naming_strategy", this.implicitNamingStrategy);
            }
        }
    }
}
  • 最后在配置LocalContainerEntityManagerFactoryBean的地方setProperties即可

多数据源事务

多数据源事务场景及其问题

  • 只涉及单库单服务事务，没问题
  • 跨库跨服务事务，有问题，当内部服务调用成功之后，而当前服务随后抛出异常，则内部服务（不同PlatformTransactionManager）不会回滚，因为是两个事务（借助数据库实现的）

解决方案

  • 弱一致性
    • 消息队列
  • 强一致性
    • XA协议，两阶段提交协议2PC、三阶段提交协议3PC是对2的优化
  • 最终一致性
    • 事务补偿（TCC）

选择哪个方案

  • 选择的几个维度：一致性强度、吞吐量、实现复杂度、架构搭建后开发的复杂度
  • 我们项目的现状：用户不多、基本都是数据库的操作、不希望开发为此额外做很多开发
  • 这个帖子我觉得说的蛮有道理的：https://www.atomikos.com/Blog/ToXAOrNotToXA
  • 所以选择XA协议

分布式事务XA协议

使用atomikos对原项目改造

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jta-atomikos</artifactId>
</dependency>

use atomikos to impl AbstractJtaPlatform

import com.atomikos.icatch.jta.UserTransactionManager;
import org.hibernate.engine.transaction.jta.platform.internal.AbstractJtaPlatform;

import javax.transaction.TransactionManager;
import javax.transaction.UserTransaction;

public class AtomikosJtaPlatform extends AbstractJtaPlatform {

    private static final long serialVersionUID = 1L;
    private UserTransactionManager utm;

    public AtomikosJtaPlatform() {
        utm = new UserTransactionManager();
    }

    @Override
    protected TransactionManager locateTransactionManager() {
        return utm;
    }

    @Override
    protected UserTransaction locateUserTransaction() {
        return utm;
    }
}

datasource

private AtomikosDataSourceBean buildAtomikosDataSourceBean(DataSourceProperties dataSourceProperties, String resourceName) throws SQLException {
    val xaDataSource = dataSourceProperties.initializeDataSourceBuilder()
            .type(MysqlXADataSource.class).build();
    xaDataSource.setPinGlobalTxToPhysicalConnection(true);
    val atomikosDataSourceBean = new AtomikosDataSourceBean();
    atomikosDataSourceBean.setXaDataSource(xaDataSource);
    atomikosDataSourceBean.setUniqueResourceName(resourceName);
    atomikosDataSourceBean.setPoolSize(10);
    return atomikosDataSourceBean;
}
  • MysqlXADataSource：主流数据库都支持XA协议，我用的是mysql，只要实现javax.sql.XADataSource即可
  • PinGlobalTxToPhysicalConnection：这个要打开否则无法使用jta事务

一个jta事务管理器

import cc.gegee.common.jpa.Constant;
import com.atomikos.icatch.jta.UserTransactionImp;
import com.atomikos.icatch.jta.UserTransactionManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.transaction.jta.JtaTransactionManager;

import javax.transaction.UserTransaction;

@Configuration
public class JtaTransactionManagerConfig {

    @Primary
    @Bean(name = Constant.TRANSACTION_MANAGER_JTA)
    public JtaTransactionManager regTransactionManager () {
        UserTransactionManager userTransactionManager = new UserTransactionManager();
        UserTransaction userTransaction = new UserTransactionImp();
        return new JtaTransactionManager(userTransaction, userTransactionManager);
    }
}

修改原先的配置(另一个类同)

  • @EnableJpaRepositories的transactionManagerRef指向JtaTransactionManager
  • LocalContainerEntityManagerFactoryBean打开JTA
import cc.gegee.common.jpa.Constant;
import com.querydsl.jpa.impl.JPAQueryFactory;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.orm.jpa.SharedEntityManagerCreator;

import javax.persistence.EntityManager;
import javax.sql.DataSource;
import java.util.Objects;

@Configuration
@EnableJpaRepositories(
        basePackages = {
                Constant.PACKAGES_1
                , Constant.PACKAGES_2
                , Constant.PACKAGES_3
                , Constant.PACKAGES_4
        },
        entityManagerFactoryRef = Constant.ENTITY_MANAGER_FACTORY_1,
        transactionManagerRef = Constant.TRANSACTION_MANAGER_JTA
)
public class JpaConfigurationPangolin {

    private final DataSource dataSource;

    private final EntityManagerFactoryBuilder builder;

    private final HibernateConfiguration hibernateConfiguration;

    public JpaConfigurationPangolin(@Qualifier(Constant.DATA_SOURCE_1) DataSource dataSource, EntityManagerFactoryBuilder builder, HibernateConfiguration hibernateConfiguration) {
        this.dataSource = dataSource;
        this.builder = builder;
        this.hibernateConfiguration = hibernateConfiguration;
    }

    @Bean(name = Constant.ENTITY_MANAGER_FACTORY_1)
    @Primary
    public LocalContainerEntityManagerFactoryBean entityManagerFactoryBean() {
        return builder
                .dataSource(dataSource)
                .properties(hibernateConfiguration.getVendorProperties(dataSource))
                .packages(
                        Constant.PACKAGES_1
                        , Constant.PACKAGES_2
                        , Constant.PACKAGES_3
                        , Constant.PACKAGES_4
                )
                .persistenceUnit(Constant.PERSISTENCE_UNIT_1)
                .jta(true)
                .build();
    }

    @Bean(name = Constant.ENTITY_MANAGER_1)
    @Primary
    public EntityManager entityManager() {
        // return SharedEntityManagerCreator.createSharedEntityManager(Objects.requireNonNull(entityManagerFactoryBean().getObject()));
        return Objects.requireNonNull(entityManagerFactoryBean().getObject()).createEntityManager();
    }

    @Bean(name = Constant.JPA_QUERY_FACTORY_1)
    @Primary
    public JPAQueryFactory jpaQueryFactory() {
        return new JPAQueryFactory(entityManager());
    }
}

使用

  • 和原来没什么区别：在service上配置@Transactional(rollbackFor = Exception.class)即可

Hibernate Session的管理问题

  • OK，到此多数据源的配置完结了，以为可以放松一段一时间了，谁知测试发现一个奇怪的现象，就是恰巧某个功能用JPA做了修改之后，使用了querydsl做了查询，将结果返回。但是查询来的值是修改之前的。此外，其他使用querydsl查询的地方永远不再更新修改后的查询结果。

一条弯路

  • 经过初步的测试排查就是querydsl和JPA有'冲突'
  • 再经过debug，每次请求JPAQueryFactory使用的session都是同一个，并且其他请求也是同一个
  • 我们知道hibernate默认开启的是一级缓存（session），由于和JPA使用的是不同的两个session，JPAQueryFactory的session一直是同一个，而JPA执行完后又没有通知JPAQueryFactory的session，导致一直查询的是历史值
  • 开始以为是JPAQueryFactory的session机制是有缓存了不会查询导致，于是打开mysql的日志系统
## 局域网服务器
ssh xxx@192.168.2.xxx
## 用的docker
docker ps
## 进入docker 容器
docker exec -it mysql8.0 /bin/bash
## 搜索my.cnf文件位置
mysql --help --verbose | grep my.cnf
vim my.cnf
## 开启log
在 my.cnf 设置 general_log = 1
## 查看
mysql -uroot -p"pwd"
show variables where variable_name like "%general_log%";
## 监视日志
tail -f xxxx.log
  • 发现有查询，但是不知道为什么，结果不更新，且仍然从缓存取值

另一角度

  • 以前单数据源是怎样的？
  • 测试了下单数据源时，同时使用JPA和querydsl，没有问题，并且使用的是同一个session
  • 单数据源和多数据源querydsl的差别仅在于EntityManager是自己创建的，这里是关键

寻找Spring是如何实例化EntityManager

  • debug JPAQueryFactory实例化时的EntityManager，发现描述为：Shared EntityManager proxy for target factory [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean@3937fd51]
  • 我们找到org.springframework.orm.jpa.SharedEntityManagerCreator，定位到它有一个关键属性synchronizedWithTransaction，最终通过SharedEntityManagerInvocationHandler动态代理实例化EntityManager，增强EntityManager
  • 我们模拟Spring单数据源创建EntityManager，如下：
@Bean(name = Constant.ENTITY_MANAGER_1)
@Primary
public EntityManager entityManager() {
    return SharedEntityManagerCreator.createSharedEntityManager(Objects.requireNonNull(entityManagerFactoryBean().getObject()));
}

后记

  • 至此我们完成了多数据源的所有配置
  • 实际上，Spring官网提供了多数据源的使用的例子，但是是没有考虑事务、session管理的问题的，或者说是分开考虑
  • EntityManager线程安全性：JPA/Hibernate Persistence Context

resources

  • GitHub - todo

reference

最后编辑于 ：
©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 158,736评论 4赞 362
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 67,167评论 1赞 291
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 108,442评论 0赞 243
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 43,902评论 0赞 204
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 52,302评论 3赞 287
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 40,573评论 1赞 216
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 31,847评论 2赞 312
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 30,562评论 0赞 197
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 34,260评论 1赞 241
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 30,531评论 2赞 245
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 32,021评论 1赞 258
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 28,367评论 2赞 253
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 33,016评论 3赞 235
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 26,068评论 0赞 8
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 26,827评论 0赞 194
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 35,610评论 2赞 274
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 35,514评论 2赞 269