Post Reply

[R&D][QUALCOMM] Using QDL, EHostDL and DIAG interfaces & features

6th January 2013, 09:42 PM   |  #1  
SouL Shadow's Avatar
OP Senior Member
Flag Stratford, CT
Thanks Meter: 296
 
460 posts
Join Date:Joined: Jun 2010
More
This thread is for the research, development and discussion of open source tools (initially Linux) to communicate with and utilize the various proprietary interfaces available on Qualcomm devices.

Initial development is centered around the MSM8660 and MSM8960 devices, but should be applicable to nearly any Qualcomm device which includes a modem and USB port. Older devices with a Serial port may also work. Components to be supported: DMSS Download Protocol (QDL mode), Streaming Download Protocol (EHostDL), and parts of other HDLC structured Qualcomm protocols.


An expanded description, examples, references, and test programs to follow shortly.


Goals
  • To provide a partial Open Source (Linux) replacement for QPST and QXDM
  • To enable the full recovery of various Android devices based on supported Qualcomm SoC's
  • To gain a better understanding of the underlying hardware in Qualcomm based Android devices


Change Log:
  • 2013-01-06
    Initial creation to consolidate OT discussions from other threads.
  • 2013-01-07
    Expanded description
    Added external thread and web links
    Added #QDL_Dev on IRC Freenode for open discussion
  • 2013-01-28
    Updated a few posts to correct prior mistakes.


Internal Thread Links
  • coming soon...


External Thread Links

External Web Links
  • Code Aurora Forum https://www.codeaurora.org/
    Home to various Open Source projects related to Qualcomm technologies.
  • Gobi https://www.codeaurora.org/contribute/projects/gobi/
    A Code Aurora Forum project fueled by Qualcomm which serves as a reference for these protocol implementations.
  • AnyClub Blog http://www.anyclub.org/
    A blog with limited yet specific information regarding Qualcomm MSM, MDM, QRD and related products. Can get technical at times and references closed source and proprietary files/programs.


Join us for live discussion in #QDL_DEV on IRC Freenode


Credits/Thanks:
  • E:V:A for various reference threads which both sparked my interest and fueled my initial research.
  • Darkspr1te for his involvement with initial and ongoing development.
  • Ralekdev for providing additional insight in to msm8960 PBL
  • .
  • Yarrimapirate for creation of JET (Jewel Evita Toolkit) which served as my first hands-on with QDL and led me down the path to here.
  • Fuses for his emmc_recover program, which gave me my first glimpse of using HDLC to communicate with a Qualcomm based phone. Also for his typically brief and discouraging posts, which in turn drives my desire to prove him wrong
  • Captain_Throwback for providing firmware zips, testing, and more bricked phones then anyone else I've met.
  • others whom I'll add as I think of them.
Last edited by SouL Shadow; 2nd February 2013 at 02:21 PM.
The Following 15 Users Say Thank You to SouL Shadow For This Useful Post: [ View ]
6th January 2013, 09:43 PM   |  #2  
SouL Shadow's Avatar
OP Senior Member
Flag Stratford, CT
Thanks Meter: 296
 
460 posts
Join Date:Joined: Jun 2010
More
Knowledge Base
Definitions:
  • PBL = Primary Boot Loader
  • SBL = Secondary Boot Loader
  • RPM = Resource and Power Management
  • TZ = Trust Zone
  • HDLC = High-level Data Link Control
  • MSM = Mobile Station Modem
  • DMSS = Dual-Mode Subscriber Station
  • QDL = Qualcomm Download
  • QHSUSB_DLOAD = Qualcomm High Speed USB Download
  • EhostDL = Emergency Host Download
  • DCN = Document Control Number, used by Qualcomm to track their thousands of documents

Qualcomm has built in to their firmware multiple methods of communication with outside "hosts" (a computer connected to the phone). Each method serves a particular function. AT commands are used to communicate with the modem while it is "online" and their multiple diagnostic protocols communicate with the modem in "offline" mode. These diagnostic protocols use HDLC (both synchronous and asynchronous) for the framing. It is a low overhead frame/packet transport which includes a 16 bit CRC for error checking, originally used over serial connections to the phone. Today these protocols are still being used over USB. Under Linux a usb-serial connection can be established by the qcserial kernel module via a /dev/ttyUSB (ex: /dev/ttyUSB0, /dev/ttyUSB1)

HDLC: A brief overview.
The basic HDLC structure is:
Each field is a multiple of 8-bits (1 byte).
HDLC uses 0x7e for the header and flag. For AsyncHDLC the header is optional, but Qualcomm always uses it. Also, the flag of one HDLC frame is allowed to be used as the header of the next frame. It also uses 0x7d as an escape for occurrences of 0x7e and 0x7d. All escaping is done after calculating the CRC and is applied to both the packet and CRC.

The packet is further broken down in to:
The packet header consists of:

The command is a 1 byte (0x00) code that determines the layout of the packet.
The parameters vary by command and specify different command specific options and the size of any data being transferred.

The CRC is generated using the standard CRC-CCITT-16 generator polynomial of: f(x)=x^16+x^12+x^5+1
Google it for more info.

Examples:
  • NO-OP: 7e 06 4e 95 7e
  • ACK: 7e 02 6a d3 7e
  • Software Version Request: 7e 0c 14 3a 7e
  • Software Version Response: 7e 0d 0f 50 42 4c 5f 44 6c 6f 61 64 56 45 52 31 2e 30 37 41 7e

Full Documentation:
  • DMSS Download Protocol: DCN 80-39912-1 Revision E
    Describes in detail the commands used with QHSUSB_DLOAD (both SBL and PBL)
  • Streaming Download Protocol: DCN 80-V5348-1 Revision J
    Describes in detail the commands used with the Flash Programmer (MPRGxxxx.hex)
  • CDMA DMSS Serial Data: DCN 80-V1294-1 Revision YP
    Describes in detail the basic commands used with the modem Diagnostic mode. This protocol supports a MASSIVE amount of extentions covered in numerous other specialized documents. There is no current plan to implement these extensions.


...more to follow...
Last edited by SouL Shadow; 2nd February 2013 at 02:23 PM.
The Following 5 Users Say Thank You to SouL Shadow For This Useful Post: [ View ]
6th January 2013, 09:44 PM   |  #3  
SouL Shadow's Avatar
OP Senior Member
Flag Stratford, CT
Thanks Meter: 296
 
460 posts
Join Date:Joined: Jun 2010
More
SPECIAL NOTE ABOUT THE NEXT POST:

If you attempt to use the msimage.mbn,YOU MUST CREATE IT USING THE SAME VERSION (or newer) FIRMWARE ALREADY ON YOUR PHONE. I'm not 100% sure if this applies to older models, but at least with msm8960 and newer.

Why?

Because, in addition to checking the signature of the image, the PBL also checks the firmware version against an efuse value for rollback prevention. If the OEM enables this feature then an older firmware will cause an error and will jump back to the last successfully loaded version of QDL mode. (ie: pbl, sbl1, etc...) This behavior has been the cause of many bricks for HTC Evo 4g LTE (jewel) owners who try to downgrade their firmware via ruu or recovery (sorry captn).

The firmware images involved are:
sbl1, sbl2, sbl3, tz and rpm.
Last edited by SouL Shadow; 11th January 2013 at 11:45 PM.
The Following 4 Users Say Thank You to SouL Shadow For This Useful Post: [ View ]
7th January 2013, 01:55 AM   |  #4  
darkspr1te's Avatar
Senior Member
Thanks Meter: 447
 
828 posts
Join Date:Joined: Sep 2012
More
DMSS And Streaming Protocol Tool
UPDATE: Code updated as of 17-01-2013, post will update to follow new code soon - Darkspr1te

First POC, Thats Proof of concept , not piece of c**p.

The concept behind this came from Soul Shadow, who like me feel that in a world without walls and fences who need windows and gates.
The original script was pulled from some git/website i dont remember belonging to a person i only know as scotty (please step forward )
JCSullins over from rootzwiki went running with the script to give us this working concept.

What is it?
This script fire's HDLC encoded frames at the serial port, namely qcserial for a Qualcomm HS_USB QDLOAD device 05c6:9008
within these frames are commands for various functions with great names like Hello, and Open MI.
Here is a example frame
Code:
0x7e 0x0a 0x63 0x74 0x7e
0x7e start of frame
0x0a command (this one is with out data)
0x63 crc low bit
0x74 crc high bit
0x7e close of frame

HDLC is all well document around the net so i wont go over it too much just yet. the important part is knowing the commands, what they do and what the payload, if any is and how that's formatted.

Why Do We need it?
The QDLOAD and EDLOAD protocols allow further control over your device, possible debrick solutions too, thats why we are developing it, some have mentioned other possible benifits but to reduce the google crew sending eveyone here looking for off-s solution and this thread going off topic we are avoiding that.Please can you also avoid topics of that nature.

What About Windows
You already have QPST and QXDM, us poor linux users dont. I am sure cygwin can help you there, some code changes may be required.

Enough Already, Gimme
https://github.com/jcsullins/qdloader


How Do I use it?
First you need to get the hex file for your device, if it's a msm8660 then your need mrpg8660.hex, they are found elsewhere, links will be posted later but for now use the search
then you need to run hex2bin on the hex file to have mrpgXXXX.bin which you rename hex.bin
then you need your emmc payload, this normally would be xxxx_msimage.mbn which you rename hex2.bin
then perl qdload.pl while you device is plugged in, there will be some debug output showing first and second stage uploads.

It's Didnt work,my device is still bricked, Answer my PM dammit!!
As I mentioned , this is a proof of concept file for study and not really ment to be a oneclick solution. Feed back is most welcome but dont mail the developers with questions for debricking the device, this is a tool to study and develop.

I REPEAT, stay away from this tool if you are not already familiar with qualcomm boot procedures, emmc system and the like.

EDIT: We have Found the original author of the script which we based the above on.
Scotty Walker
https://github.com/tmzt/g2root-kmod/...er/scotty2/pbl
Credits to The Man for making his work public.
Last edited by darkspr1te; 17th January 2013 at 10:06 PM. Reason: New Info
The Following 6 Users Say Thank You to darkspr1te For This Useful Post: [ View ]
9th January 2013, 06:09 AM   |  #5  
saketh91's Avatar
Senior Member
Flag New York
Thanks Meter: 72
 
481 posts
Join Date:Joined: Sep 2011
More
deleted
Last edited by saketh91; 12th January 2013 at 06:06 AM.
12th January 2013, 06:30 AM   |  #6  
saketh91's Avatar
Senior Member
Flag New York
Thanks Meter: 72
 
481 posts
Join Date:Joined: Sep 2011
More
Quote:
Originally Posted by SouL Shadow

SPECIAL NOTE ABOUT THE NEXT POST:

If you attempt to use the msimage.mbn,YOU MUST CREATE IT USING THE SAME VERSION (or newer) FIRMWARE ALREADY ON YOUR PHONE. I'm not 100% sure if this applies to older models, but at least with msm8960 and newer.

Why?

Because, in addition to checking the signature of the image, the PBL also checks the firmware version against an efuse value for rollback prevention. If the OEM enables this feature then an older firmware will cause an error and will jump back to the last successfully loaded version of QDL mode. (ie: pbl, sbl1, etc...) This behavior has been the cause of many bricks for HTC Evo 4g LTE (jewel) owners who try to downgrade their firmware via ruu or recovery (sorry captn).

The firmware images involved are:
sbl1, sbl2, sbl3, tz and rpm.

I was on 1.73 firmware(older or stock) when i bricked my phone.so you mean i have create a mbn file from a device which has 1.73 firmware?

and also how do you check whether a particular mbn file belongs to particular firmware only?.please help me
i have these files which i uploaded.can you see if these can be used for this method.

also i got the same error as i got before after following the post#4 method.i will soon upload the log file to you

sorry for being a noob
thanks
Attached Files
File Type: zip 8960.zip - [Click for QR Code] (446.7 KB, 468 views)
Last edited by saketh91; 12th January 2013 at 06:43 AM.
12th January 2013, 02:01 PM   |  #7  
SouL Shadow's Avatar
OP Senior Member
Flag Stratford, CT
Thanks Meter: 296
 
460 posts
Join Date:Joined: Jun 2010
More
Quote:
Originally Posted by saketh91

I was on 1.73 firmware(older or stock) when i bricked my phone.so you mean i have create a mbn file from a device which has 1.73 firmware?

Yes. All you need is the image files from an update or ruu. Check your device's forum, I'm sure someone has posted full firmware zip's. Just grab the correct one and wait for instructions.

Quote:
Originally Posted by saketh91

and also how do you check whether a particular mbn file belongs to particular firmware only?.please help me
i have these files which i uploaded.can you see if these can be used for this method.

The msimage.mbn is created from the firmware images (sbl1, sbl2, sbl3, tz, rpm) along with the partition information for that device.

Darkspr1te has been working on tools to create this file. Once he determines them to be ready, he will post them along with instructions on how to use them.

Quote:
Originally Posted by saketh91

also i got the same error as i got before after following the post#4 method.i will soon upload the log file to you

sorry for being a noob
thanks

Thank you for your patience and support. I know it's been frustrating being without your phone for so long. We try to share information as soon as we learn it. But sometimes it takes longer than expected to develop ways to utilize our newly found knowledge.

-SLS-
The Following User Says Thank You to SouL Shadow For This Useful Post: [ View ]
13th January 2013, 12:11 AM   |  #8  
Junior Member
Thanks Meter: 0
 
2 posts
Join Date:Joined: Jan 2013
Team Unlimited has (what I believe is) the stock RUU for the Evo 4g LTE for HBOOT 1.15, 1.15 and 2.09 here (EDIT: can't post links because I am a noob with under 10 posts)
Using QPST to flash MPRG8960.HEX and 8960_msimage.mbn it always fails on 'Sending Go Command 0x2A000000', which I think is the pbl authenticating sbl1? If you are right and find a way to insert the correctly signed files into the .mbn I owe you both a beer
13th January 2013, 01:15 PM   |  #9  
saketh91's Avatar
Senior Member
Flag New York
Thanks Meter: 72
 
481 posts
Join Date:Joined: Sep 2011
More
Quote:
Originally Posted by SouL Shadow

Yes. All you need is the image files from an update or ruu. Check your device's forum, I'm sure someone has posted full firmware zip's. Just grab the correct one and wait for instructions.



The msimage.mbn is created from the firmware images (sbl1, sbl2, sbl3, tz, rpm) along with the partition information for that device.

Darkspr1te has been working on tools to create this file. Once he determines them to be ready, he will post them along with instructions on how to use them.



Thank you for your patience and support. I know it's been frustrating being without your phone for so long. We try to share information as soon as we learn it. But sometimes it takes longer than expected to develop ways to utilize our newly found knowledge.

-SLS-

thanks for the reply.i will definitely wait for you to come up with solution.I am just trying to help you by providing you with logs.I have full confidence in you.I will wait for sure.thanks for all the help.
13th January 2013, 03:08 PM   |  #10  
SouL Shadow's Avatar
OP Senior Member
Flag Stratford, CT
Thanks Meter: 296
 
460 posts
Join Date:Joined: Jun 2010
More
Quote:
Originally Posted by withRandomPrecision

Team Unlimited has (what I believe is) the stock RUU for the Evo 4g LTE for HBOOT 1.15, 1.15 and 2.09 here (EDIT: can't post links because I am a noob with under 10 posts)
Using QPST to flash MPRG8960.HEX and 8960_msimage.mbn it always fails on 'Sending Go Command 0x2A000000', which I think is the pbl authenticating sbl1? If you are right and find a way to insert the correctly signed files into the .mbn I owe you both a beer

The files you refer to on Team Unlimited's site http://www.unlimited.io are the RUU's for the HTC Evo 4g LTE (jewel). For non-Htc ppl, an RUU is a windows executable that contains the full firmware and software for the given phone. Each RUU corresponds to a software release. Yes, the firmware images needed to create an msimage.mbn for jewel are contained in the RUU.

As for the mprg8960.hex:
  • The PBL does not perform OEM signature checking on the hex file. The hex file is built by Qualcomm before distributing the sources to the OEM's. It's sole function is to program blank or corrupted flash memory (nand, emmc, etc...) with the firmware bootloaders (sbl1, sbl2, sbl3, tz, rpm).
  • The address 0x2a000000 is where the mprg.hex is stored in memory. After upload the 'GO' command is used to transfer execution to the flash programmer (the hex file). The phone is supposed to acknowledge the 'GO' command before jumping to the new code. It appears that the 8960 firmware in use by HTC and Samsung has a bug and is not sending that acknowledgement. QPST waits for this acknowledgement before moving on to the next step. This is one of the reasons that prompted the creation of this thread, to develop an alternative to QPST.
  • Using the perl script posted above by Darkspr1te, other ppl have shown that the 'GO' command DOES transfer execution to the flash programmer and have used it to write the firmware (msimage.mbn) to emmc flash, but have not yet had success booting the loaded firmware. That is why I pointed out the need for the correct firmware version to be used to create the msimage.mbn.

-SLS-
Last edited by SouL Shadow; 13th January 2013 at 03:12 PM.

The Following User Says Thank You to SouL Shadow For This Useful Post: [ View ]
Post Reply Subscribe to Thread

Tags
qdl, qualcomm, r&d
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes