• Introducing XDA Computing: Discussion zones for Hardware, Software, and more!    Check it out!

[R&D][QUALCOMM] Using QDL, EHostDL and DIAG interfaces & features

Search This thread

SouL Shadow

Senior Member
Jun 17, 2010
466
325
Stratford, CT
www.soulshadow.net
This thread is for the research, development and discussion of open source tools (initially Linux) to communicate with and utilize the various proprietary interfaces available on Qualcomm devices.

Initial development is centered around the MSM8660 and MSM8960 devices, but should be applicable to nearly any Qualcomm device which includes a modem and USB port. Older devices with a Serial port may also work. Components to be supported: DMSS Download Protocol (QDL mode), Streaming Download Protocol (EHostDL), and parts of other HDLC structured Qualcomm protocols.


An expanded description, examples, references, and test programs to follow shortly.


Goals
  • To provide a partial Open Source (Linux) replacement for QPST and QXDM
  • To enable the full recovery of various Android devices based on supported Qualcomm SoC's
  • To gain a better understanding of the underlying hardware in Qualcomm based Android devices


Change Log:
  • 2013-01-06
    Initial creation to consolidate OT discussions from other threads.
  • 2013-01-07
    Expanded description
    Added external thread and web links
    Added #QDL_Dev on IRC Freenode for open discussion
  • 2013-01-28
    Updated a few posts to correct prior mistakes.


Internal Thread Links
  • coming soon...


External Thread Links


External Web Links
  • Code Aurora Forum https://www.codeaurora.org/
    Home to various Open Source projects related to Qualcomm technologies.
  • Gobi https://www.codeaurora.org/contribute/projects/gobi/
    A Code Aurora Forum project fueled by Qualcomm which serves as a reference for these protocol implementations.
  • AnyClub Blog http://www.anyclub.org/
    A blog with limited yet specific information regarding Qualcomm MSM, MDM, QRD and related products. Can get technical at times and references closed source and proprietary files/programs.


Join us for live discussion in #QDL_DEV on IRC Freenode


Credits/Thanks:
  • E:V:A for various reference threads which both sparked my interest and fueled my initial research.
  • Darkspr1te for his involvement with initial and ongoing development.
  • Ralekdev for providing additional insight in to msm8960 PBL
  • .
  • Yarrimapirate for creation of JET (Jewel Evita Toolkit) which served as my first hands-on with QDL and led me down the path to here.
  • Fuses for his emmc_recover program, which gave me my first glimpse of using HDLC to communicate with a Qualcomm based phone. Also for his typically brief and discouraging posts, which in turn drives my desire to prove him wrong :)
  • Captain_Throwback for providing firmware zips, testing, and more bricked phones then anyone else I've met.
  • others whom I'll add as I think of them.
 
Last edited:

SouL Shadow

Senior Member
Jun 17, 2010
466
325
Stratford, CT
www.soulshadow.net
Knowledge Base

Definitions:
  • PBL = Primary Boot Loader
  • SBL = Secondary Boot Loader
  • RPM = Resource and Power Management
  • TZ = Trust Zone
  • HDLC = High-level Data Link Control
  • MSM = Mobile Station Modem
  • DMSS = Dual-Mode Subscriber Station
  • QDL = Qualcomm Download
  • QHSUSB_DLOAD = Qualcomm High Speed USB Download
  • EhostDL = Emergency Host Download
  • DCN = Document Control Number, used by Qualcomm to track their thousands of documents

Qualcomm has built in to their firmware multiple methods of communication with outside "hosts" (a computer connected to the phone). Each method serves a particular function. AT commands are used to communicate with the modem while it is "online" and their multiple diagnostic protocols communicate with the modem in "offline" mode. These diagnostic protocols use HDLC (both synchronous and asynchronous) for the framing. It is a low overhead frame/packet transport which includes a 16 bit CRC for error checking, originally used over serial connections to the phone. Today these protocols are still being used over USB. Under Linux a usb-serial connection can be established by the qcserial kernel module via a /dev/ttyUSB (ex: /dev/ttyUSB0, /dev/ttyUSB1)

HDLC: A brief overview.
The basic HDLC structure is:
Each field is a multiple of 8-bits (1 byte).
HDLC uses 0x7e for the header and flag. For AsyncHDLC the header is optional, but Qualcomm always uses it. Also, the flag of one HDLC frame is allowed to be used as the header of the next frame. It also uses 0x7d as an escape for occurrences of 0x7e and 0x7d. All escaping is done after calculating the CRC and is applied to both the packet and CRC.

The packet is further broken down in to:
The packet header consists of:

The command is a 1 byte (0x00) code that determines the layout of the packet.
The parameters vary by command and specify different command specific options and the size of any data being transferred.

The CRC is generated using the standard CRC-CCITT-16 generator polynomial of: f(x)=x^16+x^12+x^5+1
Google it for more info.

Examples:
  • NO-OP: 7e 06 4e 95 7e
  • ACK: 7e 02 6a d3 7e
  • Software Version Request: 7e 0c 14 3a 7e
  • Software Version Response: 7e 0d 0f 50 42 4c 5f 44 6c 6f 61 64 56 45 52 31 2e 30 37 41 7e

Full Documentation:
  • DMSS Download Protocol: DCN 80-39912-1 Revision E
    Describes in detail the commands used with QHSUSB_DLOAD (both SBL and PBL)
  • Streaming Download Protocol: DCN 80-V5348-1 Revision J
    Describes in detail the commands used with the Flash Programmer (MPRGxxxx.hex)
  • CDMA DMSS Serial Data: DCN 80-V1294-1 Revision YP
    Describes in detail the basic commands used with the modem Diagnostic mode. This protocol supports a MASSIVE amount of extentions covered in numerous other specialized documents. There is no current plan to implement these extensions.


...more to follow...
 
Last edited:

SouL Shadow

Senior Member
Jun 17, 2010
466
325
Stratford, CT
www.soulshadow.net
SPECIAL NOTE ABOUT THE NEXT POST:

If you attempt to use the msimage.mbn,YOU MUST CREATE IT USING THE SAME VERSION (or newer) FIRMWARE ALREADY ON YOUR PHONE. I'm not 100% sure if this applies to older models, but at least with msm8960 and newer.

Why?

Because, in addition to checking the signature of the image, the PBL also checks the firmware version against an efuse value for rollback prevention. If the OEM enables this feature then an older firmware will cause an error and will jump back to the last successfully loaded version of QDL mode. (ie: pbl, sbl1, etc...) This behavior has been the cause of many bricks for HTC Evo 4g LTE (jewel) owners who try to downgrade their firmware via ruu or recovery (sorry captn).

The firmware images involved are:
sbl1, sbl2, sbl3, tz and rpm.
 
Last edited:

darkspr1te

Senior Member
Sep 24, 2012
947
583
DMSS And Streaming Protocol Tool

UPDATE: Code updated as of 17-01-2013, post will update to follow new code soon - Darkspr1te

First POC, Thats Proof of concept , not piece of c**p.

The concept behind this came from Soul Shadow, who like me feel that in a world without walls and fences who need windows and gates.
The original script was pulled from some git/website i dont remember belonging to a person i only know as scotty (please step forward )
JCSullins over from rootzwiki went running with the script to give us this working concept.

What is it?
This script fire's HDLC encoded frames at the serial port, namely qcserial for a Qualcomm HS_USB QDLOAD device 05c6:9008
within these frames are commands for various functions with great names like Hello, and Open MI.
Here is a example frame
Code:
0x7e 0x0a 0x63 0x74 0x7e

0x7e start of frame
0x0a command (this one is with out data)
0x63 crc low bit
0x74 crc high bit
0x7e close of frame

HDLC is all well document around the net so i wont go over it too much just yet. the important part is knowing the commands, what they do and what the payload, if any is and how that's formatted.

Why Do We need it?
The QDLOAD and EDLOAD protocols allow further control over your device, possible debrick solutions too, thats why we are developing it, some have mentioned other possible benifits but to reduce the google crew sending eveyone here looking for off-s solution and this thread going off topic we are avoiding that.Please can you also avoid topics of that nature.

What About Windows
You already have QPST and QXDM, us poor linux users dont. I am sure cygwin can help you there, some code changes may be required.

Enough Already, Gimme
https://github.com/jcsullins/qdloader


How Do I use it?
First you need to get the hex file for your device, if it's a msm8660 then your need mrpg8660.hex, they are found elsewhere, links will be posted later but for now use the search
then you need to run hex2bin on the hex file to have mrpgXXXX.bin which you rename hex.bin
then you need your emmc payload, this normally would be xxxx_msimage.mbn which you rename hex2.bin
then perl qdload.pl while you device is plugged in, there will be some debug output showing first and second stage uploads.

It's Didnt work,my device is still bricked, Answer my PM dammit!!
As I mentioned , this is a proof of concept file for study and not really ment to be a oneclick solution. Feed back is most welcome but dont mail the developers with questions for debricking the device, this is a tool to study and develop.

I REPEAT, stay away from this tool if you are not already familiar with qualcomm boot procedures, emmc system and the like.

EDIT: We have Found the original author of the script which we based the above on.
Scotty Walker
https://github.com/tmzt/g2root-kmod/tree/master/scotty2/pbl
Credits to The Man for making his work public.
 
Last edited:

saketh91

Senior Member
Sep 10, 2011
574
93
Denver
SPECIAL NOTE ABOUT THE NEXT POST:

If you attempt to use the msimage.mbn,YOU MUST CREATE IT USING THE SAME VERSION (or newer) FIRMWARE ALREADY ON YOUR PHONE. I'm not 100% sure if this applies to older models, but at least with msm8960 and newer.

Why?

Because, in addition to checking the signature of the image, the PBL also checks the firmware version against an efuse value for rollback prevention. If the OEM enables this feature then an older firmware will cause an error and will jump back to the last successfully loaded version of QDL mode. (ie: pbl, sbl1, etc...) This behavior has been the cause of many bricks for HTC Evo 4g LTE (jewel) owners who try to downgrade their firmware via ruu or recovery (sorry captn).

The firmware images involved are:
sbl1, sbl2, sbl3, tz and rpm.

I was on 1.73 firmware(older or stock) when i bricked my phone.so you mean i have create a mbn file from a device which has 1.73 firmware?

and also how do you check whether a particular mbn file belongs to particular firmware only?.please help me
i have these files which i uploaded.can you see if these can be used for this method.

also i got the same error as i got before after following the post#4 method.i will soon upload the log file to you

sorry for being a noob
thanks
 

Attachments

  • 8960.zip
    446.7 KB · Views: 3,985
Last edited:

SouL Shadow

Senior Member
Jun 17, 2010
466
325
Stratford, CT
www.soulshadow.net
I was on 1.73 firmware(older or stock) when i bricked my phone.so you mean i have create a mbn file from a device which has 1.73 firmware?

Yes. All you need is the image files from an update or ruu. Check your device's forum, I'm sure someone has posted full firmware zip's. Just grab the correct one and wait for instructions.

and also how do you check whether a particular mbn file belongs to particular firmware only?.please help me
i have these files which i uploaded.can you see if these can be used for this method.

The msimage.mbn is created from the firmware images (sbl1, sbl2, sbl3, tz, rpm) along with the partition information for that device.

Darkspr1te has been working on tools to create this file. Once he determines them to be ready, he will post them along with instructions on how to use them.

also i got the same error as i got before after following the post#4 method.i will soon upload the log file to you

sorry for being a noob
thanks

Thank you for your patience and support. I know it's been frustrating being without your phone for so long. We try to share information as soon as we learn it. But sometimes it takes longer than expected to develop ways to utilize our newly found knowledge.

-SLS-
 
  • Like
Reactions: withRandomPrecision

withRandomPrecision

New member
Jan 1, 2013
2
0
Team Unlimited has (what I believe is) the stock RUU for the Evo 4g LTE for HBOOT 1.15, 1.15 and 2.09 here (EDIT: can't post links because I am a noob with under 10 posts)
Using QPST to flash MPRG8960.HEX and 8960_msimage.mbn it always fails on 'Sending Go Command 0x2A000000', which I think is the pbl authenticating sbl1? If you are right and find a way to insert the correctly signed files into the .mbn I owe you both a beer :)
 

saketh91

Senior Member
Sep 10, 2011
574
93
Denver
Yes. All you need is the image files from an update or ruu. Check your device's forum, I'm sure someone has posted full firmware zip's. Just grab the correct one and wait for instructions.



The msimage.mbn is created from the firmware images (sbl1, sbl2, sbl3, tz, rpm) along with the partition information for that device.

Darkspr1te has been working on tools to create this file. Once he determines them to be ready, he will post them along with instructions on how to use them.



Thank you for your patience and support. I know it's been frustrating being without your phone for so long. We try to share information as soon as we learn it. But sometimes it takes longer than expected to develop ways to utilize our newly found knowledge.

-SLS-

thanks for the reply.i will definitely wait for you to come up with solution.I am just trying to help you by providing you with logs.I have full confidence in you.I will wait for sure.thanks for all the help.
 

SouL Shadow

Senior Member
Jun 17, 2010
466
325
Stratford, CT
www.soulshadow.net
Team Unlimited has (what I believe is) the stock RUU for the Evo 4g LTE for HBOOT 1.15, 1.15 and 2.09 here (EDIT: can't post links because I am a noob with under 10 posts)
Using QPST to flash MPRG8960.HEX and 8960_msimage.mbn it always fails on 'Sending Go Command 0x2A000000', which I think is the pbl authenticating sbl1? If you are right and find a way to insert the correctly signed files into the .mbn I owe you both a beer :)

The files you refer to on Team Unlimited's site http://www.unlimited.io are the RUU's for the HTC Evo 4g LTE (jewel). For non-Htc ppl, an RUU is a windows executable that contains the full firmware and software for the given phone. Each RUU corresponds to a software release. Yes, the firmware images needed to create an msimage.mbn for jewel are contained in the RUU.

As for the mprg8960.hex:

  • The PBL does not perform OEM signature checking on the hex file. The hex file is built by Qualcomm before distributing the sources to the OEM's. It's sole function is to program blank or corrupted flash memory (nand, emmc, etc...) with the firmware bootloaders (sbl1, sbl2, sbl3, tz, rpm).
  • The address 0x2a000000 is where the mprg.hex is stored in memory. After upload the 'GO' command is used to transfer execution to the flash programmer (the hex file). The phone is supposed to acknowledge the 'GO' command before jumping to the new code. It appears that the 8960 firmware in use by HTC and Samsung has a bug and is not sending that acknowledgement. QPST waits for this acknowledgement before moving on to the next step. This is one of the reasons that prompted the creation of this thread, to develop an alternative to QPST.
  • Using the perl script posted above by Darkspr1te, other ppl have shown that the 'GO' command DOES transfer execution to the flash programmer and have used it to write the firmware (msimage.mbn) to emmc flash, but have not yet had success booting the loaded firmware. That is why I pointed out the need for the correct firmware version to be used to create the msimage.mbn.

-SLS-
 
Last edited:

saketh91

Senior Member
Sep 10, 2011
574
93
Denver
Yes. All you need is the image files from an update or ruu. Check your device's forum, I'm sure someone has posted full firmware zip's. Just grab the correct one and wait for instructions.
-SLS-

i don't know exactly which firmware version which i was on before bricking my phone.but i definitely flashed a rooted sense rom. however i have all zips of the roms which i probably should have installed.also will this tool apply for every device(8960) even my at&t htc one x?
 

E:V:A

Inactive Recognized Developer
Dec 6, 2011
1,449
2,215
-∇ϕ
Great work!

The PBL does not perform OEM signature checking on the hex file.

How do you know this? (Other sources have claimed the opposite...)

...After upload the 'GO' command is used to transfer execution to the flash programmer (the hex file). The phone is supposed to acknowledge the 'GO' command before jumping to the new code. It appears that the 8960 firmware in use by HTC and Samsung has a bug and is not sending that acknowledgement. QPST waits for this acknowledgement before moving on to the next step.
a) This could be an effect of PBL signature check!
b) Even if not checked, they could easily have changed the acknowledgement string to anything else.
c) It could also be an effect of a blown Qfuse...
d) Are you saying that QPST is not connecting to your phone? (What QPST version are you using?)

Using the perl script posted above by Darkspr1te, other ppl have shown that the 'GO' command DOES transfer execution to the flash programmer and have used it to write the firmware (msimage.mbn) to emmc flash, ...
What other people? Do they even have the same phone?
 

SouL Shadow

Senior Member
Jun 17, 2010
466
325
Stratford, CT
www.soulshadow.net
Great work!



How do you know this? (Other sources have claimed the opposite...)
Qualcomm docs only mention verifying the hex, they say nothing about signature checking. For all we know it's simply verifying the uncorrupted download.

The hex is built by qualcomm and distributed with *other* files to the oem's/licensee's. It only needs to be changed when the actual hardware changes. The msimage.mbn is the oem specific component. There is no oem signature on the hex, however there may be a qualcomm signature or some kind of checksum to ensure it's a valid file.

a) This could be an effect of PBL signature check!
b) Even if not checked, they could easily have changed the acknowledgement string to anything else.

The acknowledgment does not contain any text. It's just a simple ACK reply.

c) It could also be an effect of a blown Qfuse...
d) Are you saying that QPST is not connecting to your phone? (What QPST version are you using?)

QPST hangs waiting for a response from the 8960 phones (htc evita, jewel, and sgs3), but other ppl (I don't know/remember who) using the above mentioned script have uploaded the hex and been able to communicate with the flash programmer. They were even able to upload the msimage.mbm. Although the .mbn used was probably the wrong build because after writing to emmc it did not boot in to the sbl. Either due to wrong files or older versions of firmware (causing a rollback error).

What other people? Do they even have the same phone?

This post: http://forum.xda-developers.com/showthread.php?p=36578082
Note the code part where it mentions 'openmulti' that command is only in the streaming download protocol which is used exclusively by the flash programmer.


EDIT 2013-01-28:
After a discussion with Ralekdev on IRC and reexamination of posted test results, it seems that the mprg8960.hex is NOT being executed. Will need to check the stored error code to see excatly why. Ralekdev was able to show me evidence of possible signature checking in the PBL. Again, we'll need to check the stored error code to confirm if that is the case. While this is a set back for msm8960 devices, it doesn't diminish the need for a full featured, open source, Linux replacement for QPST/QXDM.

-SLS-
 
Last edited:

E:V:A

Inactive Recognized Developer
Dec 6, 2011
1,449
2,215
-∇ϕ
Qualcomm docs only mention verifying the hex, they say nothing about signature checking. For all we know it's simply verifying the uncorrupted download.

Well, you should never trust Qualcomm documentation! By the time they write the documentation, there have been many changes.
Probably what you say is correct, but I'm not conviced since I haven't checked the code. It was a few months ago I was looking at this. Perhaps the HEX not checked for signature, since it's just the downloader. (But this doesn't make sense, since this would break the SecureBoot3 chain of trust.) But whatever is downloaded IS signature checked.

The acknowledgment does not contain any text. It's just a simple ACK reply.
Well, this is not how the Odin handshake looks like! There there is a short string, like "LOKE" / "ODIN" or something like that. (I don't remember it on top of my head.) So AFAIK, Odin is not working with these device, which would be an indication that they have changed the handshake. (What other kind of tools would the mobile operators use?)

QPST hangs waiting for a response from the 8960 phones (htc evita, jewel, and sgs3), but other ppl (I don't know/remember who) using the above mentioned script have uploaded the hex and been able to communicate with the flash programmer. They were even able to upload the msimage.mbm. Although the .mbn used was probably the wrong build because after writing to emmc it did not boot in to the sbl. Either due to wrong files or older versions of firmware (causing a rollback error).
Exactly.
 
Last edited:

withRandomPrecision

New member
Jan 1, 2013
2
0
Is it possible to access the bootloader output via USB UART for htc 8960 devices? Seems like this might be useful to get PBL/SBL output for a bricked device.
 

SouL Shadow

Senior Member
Jun 17, 2010
466
325
Stratford, CT
www.soulshadow.net
Well, you should never trust Qualcomm documentation! By the time they write the documentation, there have been many changes.
Probably what you say is correct, but I'm not conviced since I haven't checked the code. It was a few months ago I was looking at this. Perhaps the HEX not checked for signature, since it's just the downloader. (But this doesn't make sense, since this would break the SecureBoot3 chain of trust.) But whatever is downloaded IS signature checked.

Take a look at the creation date on the hex files in the source archive. They were created in November 2011. But that build is from a later date (I don't have it in front of me, but I think it's from april 2012). That source archive is directly from qualcomm. Why is that important? Because it shows that even with most changes to the source, the hex files don't need to be rebuilt. Besides, the flash programmer is fairly limited in what it can do. It's purpose is to rewrite the bootloaders to blank or corrupted nand/nor/emmc flash. Once written the phone will shut down and attempt to boot normally. Secure Boot only covers the boot process from power on to hardware initialization, security environment setup and finally loading appsbl. Everything after that is up to the oem to do whatever they choose. Although, interestingly enough, team unlimited was able to create a custom hboot (htc's appsbl) which will load normally even with signature checking...

Well, this is not how the Odin handshake looks like! There there is a short string, like "HELLO" / "ODIN" or something like that. (I don't remember it on top of my head.) So AFAIK, Odin is not working with these device, which would be an indication that they have changed the handshake. (What other kind of tools would the mobile operators use?)

I don't think we're in odin anymore toto!
PBL, the good bootloader of the east suddenly appears to tell us that someone dropped a brick on secure boot. Now all these little pdf's are singing, telling us to follow the HDLC road. Along the way we'll meet some interesting new people. There's QDL who lacks a brain. The Hex-man with no heart. And Streaming Download, a protocol in need of a little courage. Together we can follow the HDLC road to reach the great wizard of qualcomm and use the ruby .mbn file to return us to odin. That's when we'll awake to find Auntie ( a || h )boot and uncle recovery. Adb is there and fastboot and android too!

(don't ask, I don't know either ...)

And back in reality:

I've never used odin (in fact the first time I even heard of it was reading the Verizon SGS3 unlocking thread, which is how I discovered your thread, which lead me to here), but it's my understanding that it is a Samsung only feature that is integrated on the appsbl level, providing similar functionality to HTC's RUU mechanism. Although odin appears to be much more advanced. I've seen numerous samsung users with qualcomm hardware mention how they were stuck in qdl mode and no longer able to access odin to recover.

Now if you'll excuse me, I suddenly have the urge to listen to Dark Side of the Moon...

-SLS-
 
  • Like
Reactions: jcjcc

SouL Shadow

Senior Member
Jun 17, 2010
466
325
Stratford, CT
www.soulshadow.net
Is it possible to access the bootloader output via USB UART for htc 8960 devices? Seems like this might be useful to get PBL/SBL output for a bricked device.

Sort of. Only JTAG can access full output. Error and other diagnostic info can be read from memory using the DMSS Download Protocol or through the DIAG interface.

Under Linux all communication is done via usb serial converter kernel module qcserial and device node /dev/ttyUSBn where n = your device number reported by the kernel dmesg. This goes for any modern qualcomm device using a usb port. Older products used a proprietary serial wiring (outlined in the DMSS Serial Data ICD document 80-V1294-1) to access these same protocols.

The pbl/sbl's all share the same qdl code base. They will transmit a "magic" string over usb, waiting only a programmed amount of time for a connection.

If you mount debugfs
Code:
mount -t debugfs none_debugfs /sys/kernel/debug
and load a kernel module usbmon
Code:
modprobe usbmon
then you can access raw usb streams, either per bus or for the entire computer. There's a raw text interface at /sys/kernel/debug/usb/usbmon
There's also raw binary interface through /dev/usbmon[N]

Also, see the kernel source docs:
<kernel source>/Documentation/usb/usbmon.txt

On a bricked phone qcserial will recognise the device and a ttyUSB will become available OR if sbl3 was successfully loaded usb mass storage will provide the enumerated emmc partitions (although using them is still a work in progress, I have an idea how to properly do it. Will post details once I can test it).

To utilize the qdl usb serial interface you need to use the DMMS Download Protocol outlined in document 80-39912-1 Revision E.


On a working phone there is a usb serial interface available as well. However the qcserial kernel module is not programmed with the oem's vid/pid, so it doesn't load. I've been able to connect to it via generic serial converter:
Code:
modprobe usbserial vender=0x<vid> product=<pid>
Then disconnect and reconnect the usb cable to the phone. dmesg will show the new ttyUSB device.
Unfortunately I haven't been able to actually do anything with it yet. On a working phone it should connect you to the modem which you can use AT commands to interact with. There is also an AT command to switch to DIAG mode. From DIAG more you would use the DMSS Serial Data protocol (doc 80-V1294-1 Revision YP), another HDLC based protocol, to interact.


I have a large number of doc's covering all the above mentioned items and much more (just over 100 pdf's). Unfortunately they are all watermarked with the actual username who had access. If someone has or can point me to a program that can remove said watermarks then I would happily share all of them.

-SLS-
 
  • Like
Reactions: fairsimple

E:V:A

Inactive Recognized Developer
Dec 6, 2011
1,449
2,215
-∇ϕ
... Unfortunately they are all watermarked with the actual username who had access. If someone has or can point me to a program that can remove said watermarks then I would happily share all of them.
Did you actually try to google that?

http://www.slideshare.net/linsu39/5-solutions-to-remove-pdf-watermark
http://download.cnet.com/We-PDF-Watermark-Remover/3000-18497_4-75593137.html

http://online2pdf.com/
http://www.freepdfconvert.com/#
http://foxyutils.com/splitpdf/
 
Last edited by a moderator:

SouL Shadow

Senior Member
Jun 17, 2010
466
325
Stratford, CT
www.soulshadow.net

Hah, yes I did. These pdf's are encrypted so most tools want a password to edit them. Looking for a Linux command line utility so I can strip about 100 pdf's. Found pdftk but it requires a password to work on encrypted pdf's. I was able to convert an encrypted pdf to a non-encrypted pdf using the pdftocairo tool... but that changes the raw pdf data so finding the watermark data is more difficult. Now I'm searching for a pdf editor since my linux distro didn't come with one. Unfortunately I've spent half my day off working on this when I could have been programming.

EDIT:
found qpdf on sourceforge!
qpdf + grep + sed = fully automated bash script to clean all the pdf's :)

EDIT2:
I now have a working script to remove the watermarks. Found a few bugs while cleaning my document archive. I will post it as soon as I can work them out.

-SLS-
 
Last edited by a moderator:
  • Like
Reactions: E:V:A

viperbjk

Inactive Recognized Developer
Nov 21, 2007
401
54
Munich
Well, you should never trust Qualcomm documentation! By the time they write the documentation, there have been many changes.
Probably what you say is correct, but I'm not conviced since I haven't checked the code. It was a few months ago I was looking at this. Perhaps the HEX not checked for signature, since it's just the downloader. (But this doesn't make sense, since this would break the SecureBoot3 chain of trust.) But whatever is downloaded IS signature checked.

Well, this is not how the Odin handshake looks like! There there is a short string, like "LOKE" / "ODIN" or something like that. (I don't remember it on top of my head.) So AFAIK, Odin is not working with these device, which would be an indication that they have changed the handshake. (What other kind of tools would the mobile operators use?)

Exactly.

Absolutely correct. The download code itself has a mechanism to verify if it is valid. Some vendors check the download code before being executed if they are signed correctly, others leave the downloader as it is, but check the md5 signature within the downloader. However we managed to exploit the md5 verification to rewrite the msm7x bootloader to let us read full flash connected to radio. Not sure if they changed a lot regarding the msm89xx chipsets, but I'm going to have a look at that again, if needed. Regarding the flashing process, the flashed files are signed and checked for validity after uploading, rsa keys are in both amss and oemsbl.

Odin Protocol mainly belongs to samsung's own cpu/bootloader and has nothing to do with the qualcomm msm's/qsd's/qsc's.

What we speak of is the such called "QC Download Mode". Using the tty interface being in QC DM Mode you can just send the "3A" command to enter the "QC Download mode". For some mobiles, even if you have access to the radio download mode (qc) you cannot flash and repair the flash that belongs to the PDA part (most seen for those OMAP / MSM combinations). It's just because both cpu's use their own flash module for their firmware parts (means the flash isn't routed to both cpus, thus technically impossible).

WBR
 

Top Liked Posts

  • There are no posts matching your filters.
  • 27
    This thread is for the research, development and discussion of open source tools (initially Linux) to communicate with and utilize the various proprietary interfaces available on Qualcomm devices.

    Initial development is centered around the MSM8660 and MSM8960 devices, but should be applicable to nearly any Qualcomm device which includes a modem and USB port. Older devices with a Serial port may also work. Components to be supported: DMSS Download Protocol (QDL mode), Streaming Download Protocol (EHostDL), and parts of other HDLC structured Qualcomm protocols.


    An expanded description, examples, references, and test programs to follow shortly.


    Goals
    • To provide a partial Open Source (Linux) replacement for QPST and QXDM
    • To enable the full recovery of various Android devices based on supported Qualcomm SoC's
    • To gain a better understanding of the underlying hardware in Qualcomm based Android devices


    Change Log:
    • 2013-01-06
      Initial creation to consolidate OT discussions from other threads.
    • 2013-01-07
      Expanded description
      Added external thread and web links
      Added #QDL_Dev on IRC Freenode for open discussion
    • 2013-01-28
      Updated a few posts to correct prior mistakes.


    Internal Thread Links
    • coming soon...


    External Thread Links


    External Web Links
    • Code Aurora Forum https://www.codeaurora.org/
      Home to various Open Source projects related to Qualcomm technologies.
    • Gobi https://www.codeaurora.org/contribute/projects/gobi/
      A Code Aurora Forum project fueled by Qualcomm which serves as a reference for these protocol implementations.
    • AnyClub Blog http://www.anyclub.org/
      A blog with limited yet specific information regarding Qualcomm MSM, MDM, QRD and related products. Can get technical at times and references closed source and proprietary files/programs.


    Join us for live discussion in #QDL_DEV on IRC Freenode


    Credits/Thanks:
    • E:V:A for various reference threads which both sparked my interest and fueled my initial research.
    • Darkspr1te for his involvement with initial and ongoing development.
    • Ralekdev for providing additional insight in to msm8960 PBL
    • .
    • Yarrimapirate for creation of JET (Jewel Evita Toolkit) which served as my first hands-on with QDL and led me down the path to here.
    • Fuses for his emmc_recover program, which gave me my first glimpse of using HDLC to communicate with a Qualcomm based phone. Also for his typically brief and discouraging posts, which in turn drives my desire to prove him wrong :)
    • Captain_Throwback for providing firmware zips, testing, and more bricked phones then anyone else I've met.
    • others whom I'll add as I think of them.
    9
    Knowledge Base

    Definitions:
    • PBL = Primary Boot Loader
    • SBL = Secondary Boot Loader
    • RPM = Resource and Power Management
    • TZ = Trust Zone
    • HDLC = High-level Data Link Control
    • MSM = Mobile Station Modem
    • DMSS = Dual-Mode Subscriber Station
    • QDL = Qualcomm Download
    • QHSUSB_DLOAD = Qualcomm High Speed USB Download
    • EhostDL = Emergency Host Download
    • DCN = Document Control Number, used by Qualcomm to track their thousands of documents

    Qualcomm has built in to their firmware multiple methods of communication with outside "hosts" (a computer connected to the phone). Each method serves a particular function. AT commands are used to communicate with the modem while it is "online" and their multiple diagnostic protocols communicate with the modem in "offline" mode. These diagnostic protocols use HDLC (both synchronous and asynchronous) for the framing. It is a low overhead frame/packet transport which includes a 16 bit CRC for error checking, originally used over serial connections to the phone. Today these protocols are still being used over USB. Under Linux a usb-serial connection can be established by the qcserial kernel module via a /dev/ttyUSB (ex: /dev/ttyUSB0, /dev/ttyUSB1)

    HDLC: A brief overview.
    The basic HDLC structure is:
    Each field is a multiple of 8-bits (1 byte).
    HDLC uses 0x7e for the header and flag. For AsyncHDLC the header is optional, but Qualcomm always uses it. Also, the flag of one HDLC frame is allowed to be used as the header of the next frame. It also uses 0x7d as an escape for occurrences of 0x7e and 0x7d. All escaping is done after calculating the CRC and is applied to both the packet and CRC.

    The packet is further broken down in to:
    The packet header consists of:

    The command is a 1 byte (0x00) code that determines the layout of the packet.
    The parameters vary by command and specify different command specific options and the size of any data being transferred.

    The CRC is generated using the standard CRC-CCITT-16 generator polynomial of: f(x)=x^16+x^12+x^5+1
    Google it for more info.

    Examples:
    • NO-OP: 7e 06 4e 95 7e
    • ACK: 7e 02 6a d3 7e
    • Software Version Request: 7e 0c 14 3a 7e
    • Software Version Response: 7e 0d 0f 50 42 4c 5f 44 6c 6f 61 64 56 45 52 31 2e 30 37 41 7e

    Full Documentation:
    • DMSS Download Protocol: DCN 80-39912-1 Revision E
      Describes in detail the commands used with QHSUSB_DLOAD (both SBL and PBL)
    • Streaming Download Protocol: DCN 80-V5348-1 Revision J
      Describes in detail the commands used with the Flash Programmer (MPRGxxxx.hex)
    • CDMA DMSS Serial Data: DCN 80-V1294-1 Revision YP
      Describes in detail the basic commands used with the modem Diagnostic mode. This protocol supports a MASSIVE amount of extentions covered in numerous other specialized documents. There is no current plan to implement these extensions.


    ...more to follow...
    8
    DMSS And Streaming Protocol Tool

    UPDATE: Code updated as of 17-01-2013, post will update to follow new code soon - Darkspr1te

    First POC, Thats Proof of concept , not piece of c**p.

    The concept behind this came from Soul Shadow, who like me feel that in a world without walls and fences who need windows and gates.
    The original script was pulled from some git/website i dont remember belonging to a person i only know as scotty (please step forward )
    JCSullins over from rootzwiki went running with the script to give us this working concept.

    What is it?
    This script fire's HDLC encoded frames at the serial port, namely qcserial for a Qualcomm HS_USB QDLOAD device 05c6:9008
    within these frames are commands for various functions with great names like Hello, and Open MI.
    Here is a example frame
    Code:
    0x7e 0x0a 0x63 0x74 0x7e

    0x7e start of frame
    0x0a command (this one is with out data)
    0x63 crc low bit
    0x74 crc high bit
    0x7e close of frame

    HDLC is all well document around the net so i wont go over it too much just yet. the important part is knowing the commands, what they do and what the payload, if any is and how that's formatted.

    Why Do We need it?
    The QDLOAD and EDLOAD protocols allow further control over your device, possible debrick solutions too, thats why we are developing it, some have mentioned other possible benifits but to reduce the google crew sending eveyone here looking for off-s solution and this thread going off topic we are avoiding that.Please can you also avoid topics of that nature.

    What About Windows
    You already have QPST and QXDM, us poor linux users dont. I am sure cygwin can help you there, some code changes may be required.

    Enough Already, Gimme
    https://github.com/jcsullins/qdloader


    How Do I use it?
    First you need to get the hex file for your device, if it's a msm8660 then your need mrpg8660.hex, they are found elsewhere, links will be posted later but for now use the search
    then you need to run hex2bin on the hex file to have mrpgXXXX.bin which you rename hex.bin
    then you need your emmc payload, this normally would be xxxx_msimage.mbn which you rename hex2.bin
    then perl qdload.pl while you device is plugged in, there will be some debug output showing first and second stage uploads.

    It's Didnt work,my device is still bricked, Answer my PM dammit!!
    As I mentioned , this is a proof of concept file for study and not really ment to be a oneclick solution. Feed back is most welcome but dont mail the developers with questions for debricking the device, this is a tool to study and develop.

    I REPEAT, stay away from this tool if you are not already familiar with qualcomm boot procedures, emmc system and the like.

    EDIT: We have Found the original author of the script which we based the above on.
    Scotty Walker
    https://github.com/tmzt/g2root-kmod/tree/master/scotty2/pbl
    Credits to The Man for making his work public.
    6
    SPECIAL NOTE ABOUT THE NEXT POST:

    If you attempt to use the msimage.mbn,YOU MUST CREATE IT USING THE SAME VERSION (or newer) FIRMWARE ALREADY ON YOUR PHONE. I'm not 100% sure if this applies to older models, but at least with msm8960 and newer.

    Why?

    Because, in addition to checking the signature of the image, the PBL also checks the firmware version against an efuse value for rollback prevention. If the OEM enables this feature then an older firmware will cause an error and will jump back to the last successfully loaded version of QDL mode. (ie: pbl, sbl1, etc...) This behavior has been the cause of many bricks for HTC Evo 4g LTE (jewel) owners who try to downgrade their firmware via ruu or recovery (sorry captn).

    The firmware images involved are:
    sbl1, sbl2, sbl3, tz and rpm.