Preface
中文社区关于JOOQ源码、设计方面的文章比较少，于是花了不少时间翻看JOOQ作者Lukas Eder的博客，JOOQ的源码，
得以管中窥豹，也萌生了写本文的念头，供大家参考，希望读者能踩在我的肩膀上，看的更清楚。
有的没的
JOOQ这样一个小众的ORM工具，活跃度已经江河日下，从github的统计来看，只有Lukas老大一人在苦苦支撑。
截屏2022-10-27 下午6.59.07.png
Prerequisite
Lukas老大似乎是Martin的忠实拥趸，不管是JOOQ的一些设计，还有博客都有提及。
要顺利阅读JOOQ源码，大概需要了解如下关键词：
Fluent Interface
Progressive Interface
简单使用
我们了解下简单的使用，就开始看源码吧。
public class SqlEngine {
public static void main(String[] args) {
DSLContext context = DSL.using(SQLDialect.POSTGRES);
Field idField = field("id");
Field ageField = field("age");
Field aggeField = sum(field("salary", SQLDataType.BIGINT));
Table table = table("employee");
Condition filter = idField.gt(new Integer(1));
List<Field> fields = new ArrayList<>();
fields.add(idField);
fields.add(ageField);
fields.add(aggeField);
context.select(fields).select(fields).from(table);
SelectFinalStep query = context.select(fields).from(table).where(filter).groupBy(ageField);
String querySql = query.getSQL(ParamType.INLINED);
System.out.println(querySql);
}
}
使用静态工厂方法创建DSLContext，看名称就知道DSLContext是个大杂烩，存放的是各种配置信息。
重点看context.select(fields).from(table).where(filter).groupBy(ageField)构造出来的SelectImpl对象是怎么生成SQL的，直接看toString方法。
@Override
public String toString() {
try {
// [#8355] Subtypes may have null configuration
Configuration configuration = Tools.configuration(configuration());
return create(configuration.derive(SettingsTools.clone(configuration.settings()).withRenderFormatted(true))).renderInlined(this);
}
catch (SQLDialectNotSupportedException e) {
return "[ ... " + e.getMessage() + " ... ]";
}
}
这是AbstractQueryPart中的toString方法，create生成了DefaultDSLContext，其中的renderInlined方法如下
@Override
public String renderInlined(QueryPart part) {
return renderContext().paramType(INLINED).visit(part).render();
}
renderContext是个工厂方法，创建一个RenderContext
@Override
public RenderContext renderContext() {
return new DefaultRenderContext(configuration());
}
直接看renderContext().paramType().visit(part).render()中最后的render做了什么。
@Override
public final String render() {
String prepend = null;
String result = sql.toString();
return prepend == null ? result : prepend + result;
}
只是返回了DefaultRenderContext中的sql，可见SQL的拼装已经在visit这一步完成了。
看下DefaultRenderContext.visit(QueryPart)做了什么。
@Override
public final C visit(QueryPart part) {
if (part != null) {
// Issue start clause events
// -----------------------------------------------------------------
Clause[] clauses = Tools.isNotEmpty(visitListenersStart) ? clause(part) : null;
if (clauses != null)
for (int i = 0; i < clauses.length; i++)
start(clauses[i]);
// Perform the actual visiting, or recurse into the replacement
// -----------------------------------------------------------------
QueryPart replacement = start(part);
if (replacement != null) {
QueryPartInternal internal = (QueryPartInternal) replacement;
// If this is supposed to be a declaration section and the part isn't
// able to declare anything, then disable declaration temporarily
// We're declaring fields, but "part" does not declare fields
if (declareFields() && !internal.declaresFields()) {
boolean aliases = declareAliases();
declareFields(false);
visit0(internal);
declareFields(true);
declareAliases(aliases);
}
// We're declaring tables, but "part" does not declare tables
else if (declareTables() && !internal.declaresTables()) {
boolean aliases = declareAliases();
declareTables(false);
visit0(internal);
declareTables(true);
declareAliases(aliases);
}
// We're declaring windows, but "part" does not declare windows
else if (declareWindows() && !internal.declaresWindows()) {
declareWindows(false);
visit0(internal);
declareWindows(true);
}
// We're declaring cte, but "part" does not declare cte
else if (declareCTE() && !internal.declaresCTE()) {
declareCTE(false);
visit0(internal);
declareCTE(true);
}
else if (!castModeOverride && castMode() != CastMode.DEFAULT && !internal.generatesCast()) {
CastMode previous = castMode();
castMode(CastMode.DEFAULT);
visit0(internal);
castMode(previous);
}
// We're not declaring, or "part" can declare
else {
visit0(internal);
}
}
end(replacement);
// Issue end clause events
// -----------------------------------------------------------------
if (clauses != null)
for (int i = clauses.length - 1; i >= 0; i--)
end(clauses[i]);
}
return (C) this;
}
其中的if/else处理的都是些corner case，可以不看，核心在visit0这个抽象方法中。
@Override
protected final void visit0(QueryPartInternal internal) {
int before = bindValues.size();
internal.accept(this);
int after = bindValues.size();
// [#4650] In PostgreSQL, UDTConstants are always inlined as ROW(?, ?)
// as the PostgreSQL JDBC driver doesn't support SQLData. This
// means that the above internal.accept(this) call has already
// collected the bind variable. The same is true if custom data
// type bindings use Context.visit(Param), in case of which we
// must not collect the current Param
if (after == before && paramType != INLINED && internal instanceof Param) {
Param<?> param = (Param<?>) internal;
if (!param.isInline()) {
bindValues.add(param);
Integer threshold = settings().getInlineThreshold();
if (threshold != null && threshold > 0) {
checkForceInline(threshold);
}
else {
switch (family()) {
// [#5701] Tests were conducted with PostgreSQL 9.5 and pgjdbc 9.4.1209
case POSTGRES:
checkForceInline(32767);
break;
case SQLITE:
checkForceInline(999);
break;
default:
break;
}
}
}
}
}
这段逻辑发现，处理逻辑又从RenderContext这个上下文，回到了QueryPart实现本身。（忽略中间的那一段空白，我用的JOOQ是免费版本，猜测空白是付费版本代码被删掉了）
于是我们来到了SelectQueryImpl中的accept方法。
@Override
public final void accept(Context<?> ctx) {
Table<?> dmlTable;
// [#6583] Work around MySQL's self-reference-in-DML-subquery restriction
if (ctx.subqueryLevel() == 1
&& REQUIRES_DERIVED_TABLE_DML.contains(ctx.dialect())
&& (dmlTable = (Table<?>) ctx.data(DATA_DML_TARGET_TABLE)) != null
&& containsTable(dmlTable)) {
ctx.visit(DSL.select(asterisk()).from(asTable("t")));
}
// [#3564] Emulate DISINTCT ON queries at the top level
else if (Tools.isNotEmpty(distinctOn) && EMULATE_DISTINCT_ON.contains(ctx.dialect())) {
ctx.visit(distinctOnEmulation());
}
else {
accept0(ctx);
}
}
public final void accept0(Context<?> context) {
if (context.subqueryLevel() == 0)
context.scopeStart().data(DATA_TOP_LEVEL_CTE, new TopLevelCte());
SQLDialect dialect = context.dialect();
SQLDialect family = context.family();
// [#2791] TODO: Instead of explicitly manipulating these data() objects, future versions
// of jOOQ should implement a push / pop semantics to clearly delimit such scope.
Object renderTrailingLimit = context.data(DATA_RENDER_TRAILING_LIMIT_IF_APPLICABLE);
Object localWindowDefinitions = context.data(DATA_WINDOW_DEFINITIONS);
Name[] selectAliases = (Name[]) context.data(DATA_SELECT_ALIASES);
try {
Field<?>[] originalFields = null;
Field<?>[] alternativeFields = null;
if (selectAliases != null) {
context.data().remove(DATA_SELECT_ALIASES);
originalFields = getSelect().toArray(EMPTY_FIELD);
alternativeFields = new Field[originalFields.length];
for (int i = 0; i < originalFields.length; i++)
if (i < selectAliases.length)
alternativeFields[i] = originalFields[i].as(selectAliases[i]);
else
alternativeFields[i] = originalFields[i];
}
if (TRUE.equals(renderTrailingLimit))
context.data().remove(DATA_RENDER_TRAILING_LIMIT_IF_APPLICABLE);
// [#5127] Lazy initialise this map
if (localWindowDefinitions != null)
context.data(DATA_WINDOW_DEFINITIONS, null);
if (into != null
&& !TRUE.equals(context.data(DATA_OMIT_INTO_CLAUSE))
&& EMULATE_SELECT_INTO_AS_CTAS.contains(dialect)) {
context.data(DATA_OMIT_INTO_CLAUSE, true);
context.visit(DSL.createTable(into).as(this));
context.data().remove(DATA_OMIT_INTO_CLAUSE);
return;
}
if (with != null)
context.visit(with).formatSeparator();
else if (context.subqueryLevel() == 0)
context.scopeMarkStart(BEFORE_FIRST_TOP_LEVEL_CTE)
.scopeMarkEnd(BEFORE_FIRST_TOP_LEVEL_CTE)
.scopeMarkStart(AFTER_LAST_TOP_LEVEL_CTE)
.scopeMarkEnd(AFTER_LAST_TOP_LEVEL_CTE);
pushWindow(context);
Boolean wrapDerivedTables = (Boolean) context.data(DATA_WRAP_DERIVED_TABLES_IN_PARENTHESES);
if (TRUE.equals(wrapDerivedTables)) {
context.sql('(')
.formatIndentStart()
.formatNewLine()
.data().remove(DATA_WRAP_DERIVED_TABLES_IN_PARENTHESES);
}
switch (dialect) {
case CUBRID:
case FIREBIRD:
case MARIADB:
case MYSQL:
case POSTGRES: {
if (getLimit().isApplicable() && getLimit().withTies())
toSQLReferenceLimitWithWindowFunctions(context);
else
toSQLReferenceLimitDefault(context, originalFields, alternativeFields);
break;
}
// By default, render the dialect's limit clause
default: {
toSQLReferenceLimitDefault(context, originalFields, alternativeFields);
break;
}
}
// [#1296] [#7328] FOR UPDATE is emulated in some dialects using hints
if (forLock != null)
context.visit(forLock);
// [#1952] SQL Server OPTION() clauses as well as many other optional
// end-of-query clauses are appended to the end of a query
if (!StringUtils.isBlank(option))
context.formatSeparator()
.sql(option);
if (TRUE.equals(wrapDerivedTables))
context.formatIndentEnd()
.formatNewLine()
.sql(')')
.data(DATA_WRAP_DERIVED_TABLES_IN_PARENTHESES, true);
}
finally {
context.data(DATA_WINDOW_DEFINITIONS, localWindowDefinitions);
if (renderTrailingLimit != null)
context.data(DATA_RENDER_TRAILING_LIMIT_IF_APPLICABLE, renderTrailingLimit);
if (selectAliases != null)
context.data(DATA_SELECT_ALIASES, selectAliases);
}
if (context.subqueryLevel() == 0)
context.scopeEnd();
}
核心在这儿，toSQLReferenceLimitDefault(context, originalFields, alternativeFields);
private final void toSQLReferenceLimitDefault(Context<?> context, Field<?>[] originalFields, Field<?>[] alternativeFields) {
Object data = context.data(DATA_RENDER_TRAILING_LIMIT_IF_APPLICABLE);
context.data(DATA_RENDER_TRAILING_LIMIT_IF_APPLICABLE, true);
toSQLReference0(context, originalFields, alternativeFields);
if (data == null)
context.data().remove(DATA_RENDER_TRAILING_LIMIT_IF_APPLICABLE);
else
context.data(DATA_RENDER_TRAILING_LIMIT_IF_APPLICABLE, data);
}
跳到toSQLReference0
/**
* This method renders the main part of a query without the LIMIT clause.
* This part is common to any type of limited query
*/
@SuppressWarnings("unchecked")
private final void toSQLReference0(Context<?> context, Field<?>[] originalFields, Field<?>[] alternativeFields) {
SQLDialect family = context.family();
boolean qualify = context.qualify();
int unionOpSize = unionOp.size();
boolean unionParensRequired = false;
boolean unionOpNesting = false;
// The SQL standard specifies:
//
// <query expression> ::=
// [ <with clause> ] <query expression body>
// [ <order by clause> ] [ <result offset clause> ] [ <fetch first clause> ]
//
// Depending on the dialect and on various syntax elements, parts of the above must be wrapped in
// synthetic parentheses
boolean wrapQueryExpressionInDerivedTable;
boolean wrapQueryExpressionBodyInDerivedTable = false;
boolean applySeekOnDerivedTable = applySeekOnDerivedTable();
wrapQueryExpressionInDerivedTable = false
// // [#2995] Prevent the generation of wrapping parentheses around the
// // INSERT .. SELECT statement's SELECT because they would be
// // interpreted as the (missing) INSERT column list's parens.
// || (context.data(DATA_INSERT_SELECT_WITHOUT_INSERT_COLUMN_LIST) != null && unionOpSize > 0)
;
if (wrapQueryExpressionInDerivedTable)
context.visit(K_SELECT).sql(" *")
.formatSeparator()
.visit(K_FROM).sql(" (")
.formatIndentStart()
.formatNewLine();
// [#7459] In the presence of UNIONs and other set operations, the SEEK
// predicate must be applied on a derived table, not on the individual subqueries
wrapQueryExpressionBodyInDerivedTable |= applySeekOnDerivedTable;
if (wrapQueryExpressionBodyInDerivedTable) {
context.visit(K_SELECT).sql(' ');
context.formatIndentStart()
.formatNewLine()
.sql("t.*");
if (alternativeFields != null && originalFields.length < alternativeFields.length)
context.sql(", ")
.formatSeparator()
.declareFields(true)
.visit(alternativeFields[alternativeFields.length - 1])
.declareFields(false);
context.formatIndentEnd()
.formatSeparator()
.visit(K_FROM).sql(" (")
.formatIndentStart()
.formatNewLine();
}
// [#1658] jOOQ applies left-associativity to set operators. In order to enforce that across
// all databases, we need to wrap relevant subqueries in parentheses.
if (unionOpSize > 0) {
if (!TRUE.equals(context.data(DATA_NESTED_SET_OPERATIONS)))
context.data(DATA_NESTED_SET_OPERATIONS, unionOpNesting = unionOpNesting());
for (int i = unionOpSize - 1; i >= 0; i--) {
switch (unionOp.get(i)) {
case EXCEPT: context.start(SELECT_EXCEPT); break;
case EXCEPT_ALL: context.start(SELECT_EXCEPT_ALL); break;
case INTERSECT: context.start(SELECT_INTERSECT); break;
case INTERSECT_ALL: context.start(SELECT_INTERSECT_ALL); break;
case UNION: context.start(SELECT_UNION); break;
case UNION_ALL: context.start(SELECT_UNION_ALL); break;
}
// [#3676] There might be cases where nested set operations do not
// imply required parentheses in some dialects, but better
// play safe than sorry
unionParenthesis(
context,
'(',
alternativeFields != null ? alternativeFields : getSelect().toArray(EMPTY_FIELD),
derivedTableRequired(context, this),
unionParensRequired = unionOpNesting || unionParensRequired(context)
);
}
}
for (Table<?> table : getFrom())
registerTable(context, table);
// SELECT clause
// -------------
context.start(SELECT_SELECT)
.visit(K_SELECT).separatorRequired(true);
// [#1493] Oracle hints come directly after the SELECT keyword
if (!StringUtils.isBlank(hint))
context.sql(' ').sql(hint).separatorRequired(true);
if (Tools.isNotEmpty(distinctOn))
context.visit(K_DISTINCT_ON).sql(" (").visit(distinctOn).sql(')').separatorRequired(true);
else if (distinct)
context.visit(K_DISTINCT).separatorRequired(true);
context.declareFields(true);
// [#2335] When emulating LIMIT .. OFFSET, the SELECT clause needs to generate
// non-ambiguous column names as ambiguous column names are not allowed in subqueries
if (alternativeFields != null) {
if (wrapQueryExpressionBodyInDerivedTable && originalFields.length < alternativeFields.length)
context.visit(new SelectFieldList<>(Arrays.copyOf(alternativeFields, alternativeFields.length - 1)));
else
context.visit(new SelectFieldList<>(alternativeFields));
}
// The default behaviour
else {
context.visit(getSelectResolveUnsupportedAsterisks(context.configuration()));
}
context.declareFields(false)
.end(SELECT_SELECT);
// INTO clauses
// ------------
// [#4910] This clause (and the Clause.SELECT_INTO signal) must be emitted
// only in top level SELECTs
if (!context.subquery()
) {
context.start(SELECT_INTO);
QueryPart actualInto = (QueryPart) context.data(DATA_SELECT_INTO_TABLE);
if (actualInto == null)
actualInto = into;
if (actualInto != null
&& !TRUE.equals(context.data(DATA_OMIT_INTO_CLAUSE))
&& (SUPPORT_SELECT_INTO_TABLE.contains(context.dialect()) || !(actualInto instanceof Table))) {
context.formatSeparator()
.visit(K_INTO)
.sql(' ')
.visit(actualInto);
}
context.end(SELECT_INTO);
}
// FROM and JOIN clauses
// ---------------------
context.start(SELECT_FROM)
.declareTables(true);
// [#....] Some SQL dialects do not require a FROM clause. Others do and
// jOOQ generates a "DUAL" table or something equivalent.
// See also org.jooq.impl.Dual for details.
boolean hasFrom = !getFrom().isEmpty() || REQUIRES_FROM_CLAUSE.contains(context.dialect());
List<Condition> semiAntiJoinPredicates = null;
ConditionProviderImpl where = getWhere();
if (hasFrom) {
Object previousCollect = context.data(DATA_COLLECT_SEMI_ANTI_JOIN, true);
Object previousCollected = context.data(DATA_COLLECTED_SEMI_ANTI_JOIN, null);
TableList tablelist = getFrom();
tablelist = transformInlineDerivedTables(tablelist, where);
context.formatSeparator()
.visit(K_FROM)
.separatorRequired(true)
.visit(tablelist);
semiAntiJoinPredicates = (List<Condition>) context.data(DATA_COLLECTED_SEMI_ANTI_JOIN, previousCollected);
context.data(DATA_COLLECT_SEMI_ANTI_JOIN, previousCollect);
}
context.declareTables(false)
.end(SELECT_FROM);
// WHERE clause
// ------------
context.start(SELECT_WHERE);
if (TRUE.equals(context.data().get(BooleanDataKey.DATA_SELECT_NO_DATA)))
context.formatSeparator()
.visit(K_WHERE)
.sql(' ')
.visit(falseCondition());
else if (!where.hasWhere() && semiAntiJoinPredicates == null)
;
else {
ConditionProviderImpl actual = new ConditionProviderImpl();
if (semiAntiJoinPredicates != null)
actual.addConditions(semiAntiJoinPredicates);
if (where.hasWhere())
actual.addConditions(where.getWhere());
context.formatSeparator()
.visit(K_WHERE)
.sql(' ')
.visit(actual);
}
context.end(SELECT_WHERE);
// GROUP BY and HAVING clause
// --------------------------
context.start(SELECT_GROUP_BY);
if (grouping) {
context.formatSeparator()
.visit(K_GROUP_BY)
.separatorRequired(true);
// [#1665] Empty GROUP BY () clauses need parentheses
if (Tools.isEmpty(groupBy)) {
context.sql(' ');
// [#4292] Some dialects accept constant expressions in GROUP BY
// Note that dialects may consider constants as indexed field
// references, as in the ORDER BY clause!
if (EMULATE_EMPTY_GROUP_BY_CONSTANT.contains(context.dialect()))
context.sql('0');
// [#4447] CUBRID can't handle subqueries in GROUP BY
else if (family == CUBRID)
context.sql("1 + 0");
// [#4292] Some dialects don't support empty GROUP BY () clauses
else if (EMULATE_EMPTY_GROUP_BY_OTHER.contains(context.dialect()))
context.sql('(').visit(DSL.select(one())).sql(')');
// Few dialects support the SQL standard "grand total" (i.e. empty grouping set)
else
context.sql("()");
}
else
context.visit(groupBy);
}
context.end(SELECT_GROUP_BY);
// HAVING clause
// -------------
context.start(SELECT_HAVING);
if (getHaving().hasWhere())
context.formatSeparator()
.visit(K_HAVING)
.sql(' ')
.visit(getHaving());
context.end(SELECT_HAVING);
// QUALIFY clause
// -------------
if (getQualify().hasWhere())
context.formatSeparator()
.visit(K_QUALIFY)
.sql(' ')
.visit(getQualify());
// WINDOW clause
// -------------
context.start(SELECT_WINDOW);
if (Tools.isNotEmpty(window) && SUPPORT_WINDOW_CLAUSE.contains(context.dialect()))
context.formatSeparator()
.visit(K_WINDOW)
.separatorRequired(true)
.declareWindows(true)
.visit(window)
.declareWindows(false);
context.end(SELECT_WINDOW);
// ORDER BY clause for local subselect
// -----------------------------------
toSQLOrderBy(
context,
originalFields, alternativeFields,
false, wrapQueryExpressionBodyInDerivedTable,
orderBy, limit
);
// SET operations like UNION, EXCEPT, INTERSECT
// --------------------------------------------
if (unionOpSize > 0) {
unionParenthesis(context, ')', null, derivedTableRequired(context, this), unionParensRequired);
for (int i = 0; i < unionOpSize; i++) {
CombineOperator op = unionOp.get(i);
for (Select<?> other : union.get(i)) {
boolean derivedTableRequired = derivedTableRequired(context, other);
context.formatSeparator()
.visit(op.toKeyword(family));
if (unionParensRequired)
context.sql(' ');
else
context.formatSeparator();
unionParenthesis(context, '(', other.getSelect().toArray(EMPTY_FIELD), derivedTableRequired, unionParensRequired);
context.visit(other);
unionParenthesis(context, ')', null, derivedTableRequired, unionParensRequired);
}
// [#1658] Close parentheses opened previously
if (i < unionOpSize - 1)
unionParenthesis(context, ')', null, derivedTableRequired(context, this), unionParensRequired);
switch (unionOp.get(i)) {
case EXCEPT: context.end(SELECT_EXCEPT); break;
case EXCEPT_ALL: context.end(SELECT_EXCEPT_ALL); break;
case INTERSECT: context.end(SELECT_INTERSECT); break;
case INTERSECT_ALL: context.end(SELECT_INTERSECT_ALL); break;
case UNION: context.end(SELECT_UNION); break;
case UNION_ALL: context.end(SELECT_UNION_ALL); break;
}
}
if (unionOpNesting)
context.data().remove(DATA_NESTED_SET_OPERATIONS);
}
if (wrapQueryExpressionBodyInDerivedTable) {
context.formatIndentEnd()
.formatNewLine()
.sql(") t");
if (applySeekOnDerivedTable) {
context.formatSeparator()
.visit(K_WHERE)
.sql(' ')
.qualify(false)
.visit(getSeekCondition())
.qualify(qualify);
}
}
// ORDER BY clause for UNION
// -------------------------
try {
context.qualify(false);
toSQLOrderBy(
context,
originalFields, alternativeFields,
wrapQueryExpressionInDerivedTable, wrapQueryExpressionBodyInDerivedTable,
unionOrderBy, unionLimit
);
}
finally {
context.qualify(qualify);
}
}
这一段终于来到了SQL拼装的核心部分，我们把注释都抽出来，就能看出大致的拼装过程。
// SELECT clause -- 1882行
context.start(SELECT_SELECT)
.visit(K_SELECT).separatorRequired(true);
// FROM and JOIN clauses
context.formatSeparator()
.visit(K_FROM)
.separatorRequired(true)
.visit(tablelist);
// WHERE clause
context.formatSeparator()
.visit(K_WHERE)
.sql(' ')
.visit(actual);
// GROUP BY and HAVING clause
context.formatSeparator()
.visit(K_GROUP_BY)
.separatorRequired(true);
context.formatSeparator()
.visit(K_HAVING)
.sql(' ')
.visit(getHaving());
后面的orderBY就不继续贴了。
回顾一下
context.select(fields).from(table).where(filter).groupBy(ageField)构造了一个SelectQueryImpl
其中有select(SelectFieldList<SelectFieldOrAsterisk>), from(TableList), groupBy(QueryPartList<GroupField>)等元素，
这些元素都是QueryPart的实现，他们都有accept方法，接受RenderContext类型的参数，作用是把相关的select、from、where等部分转换成SQL，
RenderContext中持有SQL，随着context对select、from、where的visit，慢慢的，SQL就逐渐完善了，最终形成一个完整的SQL语句。
问题
QueryPart接口有很多子类，并且结构一层一层的往下，看的很晕，为什么这么设计。
00WechatIMG25.jpeg
如果看完了Progressive Interface这篇文章就知道答案了。
https://elegantcode.com/2009/03/21/progressive-interfaces/
