背景
最近公司的硬件设备需要升级程序，里面内置的是蓝牙4.0模块进行通信。产品已上市，而且出于成本考虑，以及升级的方便，不用拆机就可以升级，决定继续使用BLE来进行设备程序升级。
协议
在单片机上进行文件传输常用的协议主要有以下几个：
Xmodem
Xmodem is one of the most widely used file transfer protocols. The original Xmodem protocol uses 128-byte packets and a simple "checksum" method of error detection. A later enhancement, Xmodem-CRC, uses a more secure Cyclic Redundancy Check (CRC) method for error detection. Xmodem protocol always attempts to use CRC first. If the sender does not acknowledge the requests for CRC, the receiver shifts to the checksum mode and continues its request for transmission.
简而言之，xmodem就是一个被广泛使用的文件传输协议，最初的xmodem协议使用的是128字节包大小并使用简单的检验和来验证数据是否出错。后来就有了加强版的xmodem-crc，它使用的是crc来校验数据正确性。只是它默认的还是原本的检验和。
Xmodem-1K
Xmodem 1K is essentially Xmodem CRC with 1K (1024 byte) packets. On some systems and bulletin boards it may also be referred to as Ymodem. Some communication software programs, most notably Procomm Plus 1.x, also list Xmodem-1K as Ymodem. Procomm Plus 2.0 no longer refers to Xmodem-1K as Ymodem.
Xmodem-1K就是使用1024字节的包大小并加上CRC校验的更高级的Xmodem。在有些系统和软件里被称为Ymodem。
Ymodem
Ymodem is essentially Xmodem 1K that allows multiple batch file transfer. On some systems it is listed as Ymodem Batch.
Ymodem就是支持批量文件传输的Xmodem-1K协议。
以上这些，都类似于TCP协议，每次传输都会校验，但是速度会慢些。
Ymodem-g
Ymodem-g is a variant of Ymodem. It is designed to be used with modems that support error control. This protocol does not provide software error correction or recovery, but expects the modem to provide the service. It is a streaming protocol that sends and receives 1K packets in a continuous stream until instructed to stop. It does not wait for positive acknowledgement after each block is sent, but rather sends blocks in rapid succession. If any block is unsuccessfully transferred, the entire transfer is canceled.
Ymodem-g是Ymodem的一个变体，它支持错误校正。不过它并不自己处理，而是希望调制解调器来提供一个服务去处理。这是一个流式的协议，数据发送出去并不等待正确响应再发下一包，而是一个包接一个包地发送。只要有一个包发送出错，整个传输过程就将被取消。
显然，这个就类似于UDP协议，快，但是不保证稳定性。
Zmodem
Zmodem is generally the best protocol to use if the electronic service you are calling supports it. Zmodem has two significant features: it is extremely efficient and it provides crash recovery.
Like Ymodem-g, Zmodem does not wait for positive acknowledgement after each block is sent, but rather sends blocks in rapid succession. If a Zmodem transfer is canceled or interrupted for any reason, the transfer can be resurrected later and the previously transferred information need not be resent.
Zmodem一般被认为是最佳的协议。它有两个很大的特性，就是它非常高效并且支持出错时的自动恢复，并且不需要从头开始，就是它可以断点续传。
以上几个协议介绍来自这里。
如何选择
像我们的硬件设备升级，比较关注的有几个方面：
- 稳定性 容错率很低，一旦出错，就会比较麻烦。因为我们的设备在最初选择硬件的时候就没有预留多少空间可供备份。
- 要适合与BLE这样的设备进行交互。
综上，YModem是Xmodem的升级版，它保证稳定性。Zmodem支持快速传输，这个很好。
最终选择是：Ymodem协议。
协议定制
基于我们具体的需求，在原有的基础上加了一下前后的处理。
* MY YMODEM IMPLEMTATION
* *SENDER: ANDROID APP *------------------------------------------* RECEIVER: BLE DEVICE*
* HELLO BOOTLOADER ---------------------------------------------->*
* <---------------------------------------------------------------* C
* SOH 00 FF filename0fileSizeInByte0MD5[90] ZERO[38] CRC CRC----->*
* <---------------------------------------------------------------* ACK C
* STX 01 FE data[1024] CRC CRC ---------------------------------->*
* <---------------------------------------------------------------* ACK
* STX 02 FF data[1024] CRC CRC ---------------------------------->*
* <---------------------------------------------------------------* ACK
* ...
* ...
* <p>
* STX 08 F7 data[1000] CPMEOF[24] CRC CRC ----------------------->*
* <---------------------------------------------------------------* ACK
* EOT ----------------------------------------------------------->*
* <---------------------------------------------------------------* ACK
* SOH 00 FF ZERO[128] ------------------------------------------->*
* <---------------------------------------------------------------* ACK
* <---------------------------------------------------------------* MD5_OK
代码实现
首先梳理一下它应该具有哪些模块：
- 协议的核心实现
主要是负责数据传输过程中有关协议的部分，如在数据包上加入头，CRC，验证返回的正确性以及超时重发等。 - 一个协议工具类，封装包数据的提供
- 一个文件数据的读取模块：它是耗时任务，应该在子线程进行。
- 各种执行状态的监听
下面直接上代码
协议的核心实现Ymodem:
/**
* Created by leonxtp on 2017/9/16.
* Modified by leonxtp on 2017/9/16
*/
public class Ymodem implements FileStreamThread.DataRaderListener {
private static final int STEP_HELLO = 0x00;
private static final int STEP_FILE_NAME = 0x01;
private static final int STEP_FILE_BODY = 0x02;
private static final int STEP_EOT = 0x03;
private static final int STEP_END = 0x04;
private static int CURR_STEP = STEP_HELLO;
private static final byte ACK = 0x06; /* ACKnowlege */
private static final byte NAK = 0x15; /* Negative AcKnowlege */
private static final byte CAN = 0x18; /* CANcel character */
private static final byte ST_C = 'C';
private static final String MD5_OK = "MD5_OK";
private static final String MD5_ERR = "MD5_ERR";
private Context mContext;
private String filePath;
private String fileNameString = "LPK001_Android";
private String fileMd5String = "63e7bb6eed1de3cece411a7e3e8e763b";
private YModemListener listener;
private TimeOutHelper timerHelper = new TimeOutHelper();
private FileStreamThread streamThread;
//bytes has been sent of this transmission
private int bytesSent = 0;
//package data of current sending, used for int case of fail
private byte[] currSending = null;
private int packageErrorTimes = 0;
private static final int MAX_PACKAGE_SEND_ERROR_TIMES = 5;
//the timeout interval for a single package
private static final int PACKAGE_TIME_OUT = 6000;
/**
* Construct of the YModemBLE,you may don't need the fileMD5 checking,remove it
*
* @param filePath absolute path of the file
* @param fileNameString file name for sending to the terminal
* @param fileMd5String md5 for terminal checking after transmission finished
* @param listener
*/
public Ymodem(Context context, String filePath,
String fileNameString, String fileMd5String,
YModemListener listener) {
this.filePath = filePath;
this.fileNameString = fileNameString;
this.fileMd5String = fileMd5String;
this.mContext = context;
this.listener = listener;
}
/**
* Start the transmission
*/
public void start() {
sayHello();
}
/**
* Stop the transmission when you don't need it or shut it down in accident
*/
public void stop() {
bytesSent = 0;
currSending = null;
packageErrorTimes = 0;
if (streamThread != null) {
streamThread.release();
}
timerHelper.stopTimer();
}
/**
* Method for the outer caller when received data from the terminal
*/
public void onReceiveData(byte[] respData) {
//Stop the package timer
timerHelper.stopTimer();
if (respData != null && respData.length > 0) {
switch (CURR_STEP) {
case STEP_HELLO:
handleHello(respData);
break;
case STEP_FILE_NAME:
handleFileName(respData);
break;
case STEP_FILE_BODY:
handleFileBody(respData[0]);
break;
case STEP_EOT:
handleEOT(respData);
break;
case STEP_END:
handleEnd(respData);
break;
default:
break;
}
} else {
L.f("The terminal do responsed something, but received nothing??");
}
}
/**
* ==============================================================================
* Methods for sending data begin
* ==============================================================================
*/
private void sayHello() {
streamThread = new FileStreamThread(mContext, filePath, this);
CURR_STEP = STEP_HELLO;
L.f("sayHello!!!");
byte[] hello = YModemUtil.getYModelHello();
if (listener != null) {
listener.onDataReady(hello);
}
}
private void sendFileName() {
CURR_STEP = STEP_FILE_NAME;
L.f("sendFileName");
try {
int fileByteSize = streamThread.getFileByteSize();
byte[] hello = YModemUtil.getFileNamePackage(fileNameString, fileByteSize
, fileMd5String);
if (listener != null) {
listener.onDataReady(hello);
}
} catch (IOException e) {
e.printStackTrace();
}
}
private void startSendFileData() {
CURR_STEP = STEP_FILE_BODY;
L.f("startSendFileData");
streamThread.start();
}
//Callback from the data reading thread when a data package is ready
@Override
public void onDataReady(byte[] data) {
if (listener != null) {
currSending = data;
//Start the timer, it will be cancelled when reponse received,
// or trigger the timeout and resend the current package data
timerHelper.startTimer(timeoutListener, PACKAGE_TIME_OUT);
listener.onDataReady(data);
}
}
private void sendEOT() {
CURR_STEP = STEP_EOT;
L.f("sendEOT");
if (listener != null) {
listener.onDataReady(YModemUtil.getEOT());
}
}
private void sendEND() {
CURR_STEP = STEP_END;
L.f("sendEND");
if (listener != null) {
try {
listener.onDataReady(YModemUtil.getEnd());
} catch (IOException e) {
e.printStackTrace();
}
}
}
/**
* ==============================================================================
* Method for handling the response of a package
* ==============================================================================
*/
private void handleHello(byte[] value) {
int character = value[0];
if (character == ST_C) {//Receive "C" for "HELLO"
packageErrorTimes = 0;
sendFileName();
} else {
handleOthers(character);
}
}
//The file name package was responsed
private void handleFileName(byte[] value) {
if (value.length == 2 && value[0] == ACK && value[1] == ST_C) {//Receive 'ACK C' for file name
packageErrorTimes = 0;
startSendFileData();
} else if (value[0] == ST_C) {//Receive 'C' for file name, this package should be resent
handlePackageFail();
} else {
handleOthers(value[0]);
}
}
private void handleFileBody(int character) {
if (character == ACK) {//Receive ACK for file data
packageErrorTimes = 0;
bytesSent += currSending.length;
try {
if (listener != null) {
listener.onProgress(bytesSent, streamThread.getFileByteSize());
}
} catch (IOException e) {
e.printStackTrace();
}
streamThread.keepReading();
} else if (character == ST_C) {
//Receive C for file data, the ymodem cannot handle this circumstance, transmission failed...
if (listener != null) {
listener.onFailed();
}
} else {
handleOthers(character);
}
}
private void handleEOT(byte[] value) {
if (value[0] == ACK) {
packageErrorTimes = 0;
sendEND();
} else if (value[0] == ST_C) {//As we haven't received ACK, we should resend EOT
handlePackageFail();
} else {
handleOthers(value[0]);
}
}
private void handleEnd(byte[] character) {
if (character[0] == ACK) {//The last ACK represents that the transmission has been finished, but we should validate the file
packageErrorTimes = 0;
} else if ((new String(character)).equals(MD5_OK)) {//The file data has been checked,Well Done!
stop();
if (listener != null) {
listener.onSuccess();
}
} else if ((new String(character)).equals(MD5_ERR)) {//Oops...Transmission Failed...
stop();
if (listener != null) {
listener.onFailed();
}
} else {
handleOthers(character[0]);
}
}
private void handleOthers(int character) {
if (character == NAK) {//We need to resend this package as the terminal failed when checking the crc
handlePackageFail();
} else if (character == CAN) {//Some big problem occurred, transmission failed...
stop();
}
}
//Handle a failed package data ,resend it up to MAX_PACKAGE_SEND_ERROR_TIMES times.
//If still failed, then the transmission failed.
private void handlePackageFail() {
packageErrorTimes++;
if (packageErrorTimes < MAX_PACKAGE_SEND_ERROR_TIMES) {
if (listener != null) {
listener.onDataReady(currSending);
}
} else {
//Still, we stop the transmission, release the resources
stop();
if (listener != null) {
listener.onFailed();
}
}
}
/* The InputStream data reading thread was done */
@Override
public void onFinish() {
sendEOT();
}
//The timeout listener
private TimeOutHelper.ITimeOut timeoutListener = new TimeOutHelper.ITimeOut() {
@Override
public void onTimeOut() {
if (currSending != null) {
handlePackageFail();
}
}
};
public static class Builder {
private Context context;
private String filePath;
private String fileNameString;
private String fileMd5String;
private YModemListener listener;
public Builder with(Context context) {
this.context = context;
return this;
}
public Builder filePath(String filePath) {
this.filePath = filePath;
return this;
}
public Builder fileName(String fileName) {
this.fileNameString = fileName;
return this;
}
public Builder checkMd5(String fileMd5String) {
this.fileMd5String = fileMd5String;
return this;
}
public Builder callback(YModemListener listener) {
this.listener = listener;
return this;
}
public Ymodem build() {
return new Ymodem(context, filePath, fileNameString, fileMd5String, listener);
}
}
}
协议包工具类：
/**
* Util for encapsulating data package of ymodem protocol
* <p>
* Created by leonxtp on 2017/9/16.
* Modified by leonxtp on 2017/9/16
*/
public class YModemUtil {
/*This is my concrete ymodem start signal, customise it to your needs*/
private static final String HELLO = "HELLO BOOTLOADER";
private static final byte SOH = 0x01; /* Start Of Header with data size :128*/
private static final byte STX = 0x02; /* Start Of Header with data size : 1024*/
private static final byte EOT = 0x04; /* End Of Transmission */
private static final byte CPMEOF = 0x1A;/* Fill the last package if not long enough */
private static CRC16 crc16 = new CRC16();
/**
* Get the first package data for hello with a terminal
*/
public static byte[] getYModelHello() {
return HELLO.getBytes();
}
/**
* Get the file name package data
*
* @param fileNameString file name in String
* @param fileByteSize file byte size of int value
* @param fileMd5String the md5 of the file in String
*/
public static byte[] getFileNamePackage(String fileNameString,
int fileByteSize,
String fileMd5String) throws IOException {
byte seperator = 0x0;
String fileSize = fileByteSize + "";
byte[] byteFileSize = fileSize.getBytes();
byte[] fileNameBytes1 = concat(fileNameString.getBytes(),
new byte[]{seperator},
byteFileSize);
byte[] fileNameBytes2 = Arrays.copyOf(concat(fileNameBytes1,
new byte[]{seperator},
fileMd5String.getBytes()), 128);
byte seq = 0x00;
return getDataPackage(fileNameBytes2, 128, seq);
}
/**
* Get a encapsulated package data block
*
* @param block byte data array
* @param dataLength the actual content length in the block without 0 filled in it.
* @param sequence the package serial number
* @return a encapsulated package data block
*/
public static byte[] getDataPackage(byte[] block, int dataLength, byte sequence) throws IOException {
byte[] header = getDataHeader(sequence, block.length == 1024 ? STX : SOH);
//The last package, fill CPMEOF if the dataLength is not sufficient
if (dataLength < block.length) {
int startFil = dataLength;
while (startFil < block.length) {
block[startFil] = CPMEOF;
startFil++;
}
}
//We should use short size when writing into the data package as it only needs 2 bytes
short crc = (short) crc16.calcCRC(block);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(baos);
dos.writeShort(crc);
dos.close();
byte[] crcBytes = baos.toByteArray();
return concat(header, block, crcBytes);
}
/**
* Get the EOT package
*/
public static byte[] getEOT() {
return new byte[]{EOT};
}
/**
* Get the Last package
*/
public static byte[] getEnd() throws IOException {
byte seq = 0x00;
return getDataPackage(new byte[128], 128, seq);
}
/**
* Get InputStream from Assets, you can customize it from the other sources
*
* @param fileAbsolutePath absolute path of the file in asstes
*/
public static InputStream getInputStream(Context context, String fileAbsolutePath) throws IOException {
return new InputStreamSource().getStream(context, fileAbsolutePath);
}
private static byte[] getDataHeader(byte sequence, byte start) {
//The serial number of the package increases Cyclically up to 256
byte modSequence = (byte) (sequence % 0x256);
byte complementSeq = (byte) ~modSequence;
return concat(new byte[]{start},
new byte[]{modSequence},
new byte[]{complementSeq});
}
private static byte[] concat(byte[] a, byte[] b, byte[] c) {
int aLen = a.length;
int bLen = b.length;
int cLen = c.length;
byte[] concated = new byte[aLen + bLen + cLen];
System.arraycopy(a, 0, concated, 0, aLen);
System.arraycopy(b, 0, concated, aLen, bLen);
System.arraycopy(c, 0, concated, aLen + bLen, cLen);
return concated;
}
}
文件数据读取类
/**
* Thread for reading input Stream and encapsulating into a ymodem package
* <p>
* Created by leonxtp on 2017/9/16.
* Modified by leonxtp on 2017/9/16
*/
public class FileStreamThread extends Thread {
private Context mContext;
private InputStream inputStream = null;
private DataRaderListener listener;
private String filePath;
private AtomicBoolean isDataAcknowledged = new AtomicBoolean(false);
private boolean isKeepRunning = false;
private int fileByteSize = 0;
public FileStreamThread(Context mContext, String filePath, DataRaderListener listener) {
this.mContext = mContext;
this.filePath = filePath;
this.listener = listener;
}
public int getFileByteSize() throws IOException {
if (fileByteSize == 0 || inputStream == null) {
initStream();
}
return fileByteSize;
}
@Override
public void run() {
try {
prepareData();
} catch (IOException e) {
e.printStackTrace();
}
}
private void prepareData() throws IOException {
initStream();
byte[] block = new byte[1024];
int dataLength;
byte blockSequence = 1;//The data package of a file is actually started from 1
isDataAcknowledged.set(true);
isKeepRunning = true;
while (isKeepRunning) {
if (!isDataAcknowledged.get()) {
try {
//We need to sleep for a while as the sending 1024 bytes data from ble would take several seconds
//In my circumstances, this can be up to 3 seconds.
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
continue;
}
if ((dataLength = inputStream.read(block)) == -1) {
L.f("The file data has all been read...");
if (listener != null) {
onStop();
listener.onFinish();
}
break;
}
byte[] packige = YModemUtil.getDataPackage(block, dataLength, blockSequence);
if (listener != null) {
listener.onDataReady(packige);
}
blockSequence++;
isDataAcknowledged.set(false);
}
}
/**
* When received response from the terminal ,we should keep the thread keep going
*/
public void keepReading() {
isDataAcknowledged.set(true);
}
public void release() {
onStop();
listener = null;
}
private void onStop() {
isKeepRunning = false;
isDataAcknowledged.set(false);
fileByteSize = 0;
onReadFinished();
}
private void initStream() {
if (inputStream == null) {
try {
inputStream = YModemUtil.getInputStream(mContext, filePath);
fileByteSize = inputStream.available();
} catch (IOException e) {
e.printStackTrace();
}
}
}
private void onReadFinished() {
if (inputStream != null) {
try {
inputStream.close();
inputStream = null;
} catch (IOException e) {
e.printStackTrace();
}
}
}
public interface DataRaderListener {
void onDataReady(byte[] data);
void onFinish();
}
}
各种状态监听接口
/**
* Listener of the transmission process
*/
public interface YModemListener {
/* the data package has been encapsulated */
void onDataReady(byte[] data);
/*just the file data progress*/
void onProgress(int currentSent, int total);
/* the file has been correctly sent to the terminal */
void onSuccess();
/* the task has failed with several remedial measures like retrying some times*/
void onFailed();
}
具体使用
初始化
ymodem = new Ymodem.Builder()
.with(this)
.filePath("assets://demo.bin")
.fileName("demo.bin")
.checkMd5("lsfjlhoiiw121241l241lgljaf")
.callback(new YModemListener() {
@Override
public void onDataReady(byte[] data) {
//send this data[] to your ble component here...
}
@Override
public void onProgress(int currentSent, int total) {
//the progress of the file data has transmitted
}
@Override
public void onSuccess() {
//we are well done with md5 checked
}
@Override
public void onFailed() {
//the task has failed for several times of trying
}
}).build();
ymodem.start();
开始传输
ymodem.start();
当接收到设备响应
ymodem.onReceiveData(data);
停止
ymodem.stop();
完整代码在Github上。
如果对您有帮助，欢迎star、fork！
参考
Wikipedia YMODEM
xmodem、ymodem、zmodem
aesirot ymodem on github
