大家好，我是帅气小伙。今天为大家分享一下有关kettle需要导入第三方jar包时，动态加载的经验。我们在使用kettle的时候，经常需要用一些第三方的jar包，例如 mssql的驱动，我们平常可以把它放到kettle_home的lib目录下，然后重启。这样其实挺不方便的，尤其是对于集成了kettle调度的应用来说。又是往往就是一个jar包的问题，导致平台要重启N次，显示会对一些生成环境的业务造成巨大的影响。于是实现kettle运行时jar包的动态加载还是有价值的。
思路
- 为什么kettle客户端重启就能加载到lib的jar包呢？
- 1.kettle的入口 spoon.bat/spoon.sh
if %STARTTITLE%!==! SET STARTTITLE="Spoon"
REM Eventually call java instead of javaw and do not run in a separate window
if not "%SPOON_CONSOLE%"=="1" set SPOON_START_OPTION=start %STARTTITLE%
@echo on
%SPOON_START_OPTION% "%_PENTAHO_JAVA%" %OPT% -jar launcher\pentaho-application-launcher-5.4.0.1-130.jar -lib ..\%LIBSPATH% %_cmdline%
@echo off
if "%SPOON_PAUSE%"=="1" pause
显然，kettle的入口程序是 pentaho-application-launcher-5.4.0.1-130.jar，然后传入了lib参数
- 2.反编译 pentaho-application-launcher-5.4.0.1-130.jar
public static void main(String[] args) throws Exception {
Parameters parameters = Parameters.fromArgs(args, System.err);
URL location = Launcher.class.getProtectionDomain().getCodeSource().getLocation();
File appDir = FileUtil.computeApplicationDir(location, new File("."), System.err);
File configurationFile = new File(appDir, "launcher.properties");
Properties configProperties = new Properties();
try {
configProperties.load(new FileReader(configurationFile));
} catch (Exception var13) {
;
}
Configuration configuration = Configuration.create(configProperties, appDir, parameters);
if (configuration.isUninstallSecurityManager()) {
System.setSecurityManager((SecurityManager)null);
}
Iterator i$ = configuration.getSystemProperties().entrySet().iterator();
while(i$.hasNext()) {
Entry<String, String> systemProperty = (Entry)i$.next();
System.setProperty((String)systemProperty.getKey(), (String)systemProperty.getValue());
}
List<URL> jars = FileUtil.populateClasspath(configuration.getClasspath(), appDir, System.err);
jars.addAll(FileUtil.populateLibraries(configuration.getLibraries(), appDir, System.err));
URL[] classpathEntries = (URL[])((URL[])jars.toArray(new URL[jars.size()]));
ClassLoader cl = new URLClassLoader(classpathEntries);
Thread.currentThread().setContextClassLoader(cl);
if (StringUtil.isEmpty(configuration.getMainClass())) {
System.err.println("Invalid main-class entry, cannot proceed.");
System.err.println("Application Directory: " + appDir);
System.exit(1);
}
if (configuration.isDebug()) {
System.out.println("Application Directory: " + appDir);
for(int i = 0; i < classpathEntries.length; ++i) {
URL url = classpathEntries[i];
System.out.println("ClassPath[" + i + "] = " + url);
}
}
Class<?> mainClass = cl.loadClass(configuration.getMainClass());
String[] newArgs = new String[args.length - parameters.getParsedArgs()];
System.arraycopy(args, parameters.getParsedArgs(), newArgs, 0, newArgs.length);
Method method = mainClass.getMethod("main", String[].class);
method.invoke((Object)null, newArgs);
}
- 3.关键性代码
List<URL> jars = FileUtil.populateClasspath(configuration.getClasspath(), appDir, System.err);
jars.addAll(FileUtil.populateLibraries(configuration.getLibraries(), appDir, System.err));
URL[] classpathEntries = (URL[])((URL[])jars.toArray(new URL[jars.size()]));
ClassLoader cl = new URLClassLoader(classpathEntries);
Thread.currentThread().setContextClassLoader(cl);
也就是说，只需把jar包动态加载到Thread.currentThread()的classpath就可以了。
实现方式
private static ConcurrentHashMap jarMaps = new ConcurrentHashMap();//防止重复加载
/**
* 动态加载jar到kettle运行时classpath
* @param libs
* @return
*/
public static List<String> loadExtLib(List<String> libs){
List<String> success = new ArrayList<String>();
for(String path : libs){
ClassLoader cl = Thread.currentThread().getContextClassLoader();
try {
//反射调用 addURL
Method addURL = URLClassLoader.class.getDeclaredMethod("addURL", new Class[] {URL.class});
addURL.setAccessible(true);
File file = new File(path);
List<String> paths = new ArrayList<String>();
findJars(file,paths);//寻找路径下的jar包
for(String jarPath : paths){
File jar = new File(jarPath);
FileInputStream inputStream = new FileInputStream(jar);
String jarHash = md5HashCode(inputStream);
if(!jarMaps.containsKey(jarHash)){
jarMaps.put(jarHash,jarPath);
URL url = jar.toURI().toURL();
addURL.invoke(cl, new Object[] { url });
success.add(jar.getName());
log.info("成功加载："+jarPath);
}
inputStream.close();
}
} catch (Exception e) {
e.printStackTrace();
log.error("加载kettle类库失败,原因",path,e);
}
}
return success;
}
/**
* 遍历文件jar包
* @param file 文件夹
* @param jarPaths jar包路径集合
*/
public static void findJars(File file,List<String> jarPaths){
File[] files = file.listFiles();
for(File a :files){
String path = a.getPath();
if(a.isFile() && (path.endsWith(".jar") || path.endsWith(".zip"))){
jarPaths.add(path);
}
if(a.isDirectory()){
findJars(a,jarPaths);
}
}
}
/**
* java获取文件的md5值
* @param fis 输入流
* @return
*/
public static String md5HashCode(InputStream fis) {
try {
//拿到一个MD5转换器,如果想使用SHA-1或SHA-256，则传入SHA-1,SHA-256
MessageDigest md = MessageDigest.getInstance("MD5");
//分多次将一个文件读入，对于大型文件而言，比较推荐这种方式，占用内存比较少。
byte[] buffer = new byte[1024];
int length = -1;
while ((length = fis.read(buffer, 0, 1024)) != -1) {
md.update(buffer, 0, length);
}
fis.close();
//转换并返回包含16个元素字节数组,返回数值范围为-128到127
byte[] md5Bytes = md.digest();
BigInteger bigInt = new BigInteger(1, md5Bytes);//1代表绝对值
return bigInt.toString(16);//转换为16进制
} catch (Exception e) {
e.printStackTrace();
return "";
}
}
写个controller调用下就ok了。
@RequestMapping("/load/lib")
@ResponseBody
public List<String> loadLib(){
List<String> success = KettleUtils.loadExtLib(config.getLibs());
return success;
}
总结
就说这么多，有关于kettle调度这个项目，我尽量把它完善下吧，等一个spring-boot-starter-kettle吧。
