[Q] Is there an easy way to apply the patch files from the OTA on a PC?

[gee one]

I'm looking for a way to apply the patch files to the TF system image on a PC. This is not about how to get the ota to flash- I got that covered. This is about repacking roms or making flashable updates that don't rely on stock firmware.

I found a few ways, but none of them seem "easy."

The dirty way- apply the OTA and grab the system image.

From the PC command line - Linux has diff/patch, but that doesn't work for some of the patches in "IMGDIFF2" format. Besides, I'm not sure I ever got the regular BSDIFF40 patches to work.

The git option- There is the apply_patch source code in github. This works great, but I have to run it on the TF. Is it possible to compile this to work on my ubuntu box?

Maybe there is some magic script out there called apply_ota?

Thanks in advance

 

[hachamacha]

I'm looking for a way to apply the patch files to the TF system image on a PC. This is not about how to get the ota to flash- I got that covered. This is about repacking roms or making flashable updates that don't rely on stock firmware.

I found a few ways, but none of them seem "easy."

The dirty way- apply the OTA and grab the system image.

From the PC command line - Linux has diff/patch, but that doesn't work for some of the patches in "IMGDIFF2" format. Besides, I'm not sure I ever got the regular BSDIFF40 patches to work.

The git option- There is the apply_patch source code in github. This works great, but I have to run it on the TF. Is it possible to compile this to work on my ubuntu box?

Maybe there is some magic script out there called apply_ota?

Thanks in advance

I recently wrote some code for a ARM basestation (basically like a Freescale cell-phone board), and had to be able to supply some updates over time. Before the project ended, I'd also found bsdiff and bspatch and tested them on linux, where they worked fine (as long as you'd created either an ARM bsdiff/patch version or somehow took the big/little endianness of the hardware into account).

At first I thought it wasn't working but it seemed to do OK on standard cross-compiled-for-ARM files. (well, they still ran and performed the same stunt..).

The thing that worried me about this method is that they'd never have the same md5sum before as after, likely because they had a different amount of bytes. (e.g. you make your own before and after and build each, then create the patch file using bsdiff, then patch the old file using the patch file, and create a 'newtestfile', but look at the number of bytes (or md5sum it) difference between the original stage 2 file and the stage 1 + the patch file via bspatch).

Sorry that is pretty confusing the way I just tried to explain it.. I'll look this over again tomorrow and figure out if I said anything worthwhile

 

[gee one]

For inputs, the applypatch binary takes the file names, sha1 checksums of the before and after, and the final target size. Chance are high that the patch would only work if everything went in and came out properly.

apply_patch("/system/framework/framework-res.apk", "-",
            99b382b5886e505508fa3f730d778a1bed4bc04e, 9761707,
            a4798fb8d61d08b6f067e522e67107f6dd556148, package_extract_file("patch/system/framework/framework-res.apk.p"));

I think that bsdiff/patch will generally work on most of the OTA (BSDIFF40 files), but some of the files are IMGDIFF2, which targets binary/data files.

The applypatch binary works on both and as far as I can tell, is the same binary that the TF uses. Here is a snippet from the AOSP recovery... TF binary included Use the binary on your TF, not applypatch.sh! I suppose you can put it in system/xbin/

It would resolve the issue if I could compile this to run on my desktop from ubuntu. Short of that, I'm working on a script for the TF to parse the updater script.

 

[hachamacha]

For inputs, the applypatch binary takes the file names, sha1 checksums of the before and after, and the final target size. Chance are high that the patch would only work if everything went in and came out properly.

apply_patch("/system/framework/framework-res.apk", "-",
            99b382b5886e505508fa3f730d778a1bed4bc04e, 9761707,
            a4798fb8d61d08b6f067e522e67107f6dd556148, package_extract_file("patch/system/framework/framework-res.apk.p"));

I think that bsdiff/patch will generally work on most of the OTA (BSDIFF40 files), but some of the files are IMGDIFF2, which targets binary/data files.

The applypatch binary works on both and as far as I can tell, is the same binary that the TF uses. Here is a snippet from the AOSP recovery... TF binary included Use the binary on your TF, not applypatch.sh! I suppose you can put it in system/xbin/

It would resolve the issue if I could compile this to run on my desktop from ubuntu. Short of that, I'm working on a script for the TF to parse the updater script.

I'm not sure I understand the intent. I've just awoken and that's probably part of the problem..

'applypatch' is part of the standard (say 10.04 ubuntu) repo and is in a package called makepatch. Did you want to be able to run it "from" a standard x86 Ubuntu dist but have it targeting "armle" files or just be able to run it both on Ubuntu & Android correctly?

The files I tested were a) both binaries in different stages of evolution, and b) both built for armle(droids) and x86_64 and didn't behave differently, worked, and the checksum could be pre-calculated.

I'm going to mess around with applypatch when I'm out of my morning coma and get back to this in a bit.

Hah -- I do wonder if there's already a script (maybe it's a binary though) that lives in the dm* app on the tf101 that just does this already. You may have been right about that. Something has to apply those OTA patches. There's a few mechanisms for detecting them but I hadn't thought about how they're applied.

I'm trying to think if there's any scenario where the market applies 'patches' rather than always downloads the next version in entirety. Not sure about that.

Anyway, I'll see if I can get anywhere with applypatch.

Edit:: After a little messing around :

I finally see what you are trying to do and why what I'm doing is totally useless for that. I'm going to spend a little time looking over the imgdiff2 (apk) deal, and probably end up now figuring out a thing.

 

[gee one]

I'll look into the makepatch package later today. Maybe that might be what I'm looking for.

Just to be clear, the intent is to be able to patch a "stock" system without it actually running on the TF. For example, to take the stock 8.6.5.9 image, decompress system/ to the PC, apply the patches from the OTA, and then end up with a stock 8.6.5.13 image. This would all happen on the PC, not the TF. It would streamline the ROM cooking process and make it easier and faster to spit out a complete, flashable update that won't fail sha1 checks because of mods.



sent from my cyanogen(mod) vision

 

[gee one]

Zombie Thread Alert!!!

You have been warned....

OK, so my C-fu isn't strong enough to figure out how to cross compile the apply patch binary to run on a x86 computer. However, I did notice that the applypatch binary gets built in the aosp/generic_x86 build, specifically in the system/bin folder.

This will be an exectuble file that will run on a x86 linux based system and do the same thing that the applypatch binary does to the ota package during the ota update. In other words, you can apply the ota patch to a system image from the comfort of the linux command line without having to actually flash the ota. This will allow you to be the first one on your block to have the new system image without having to mess up whatever rom you are currently running.

Of course, this might all be moot if we don't see another ota for our tf. Come on ASUS, just one more so I can test this out on a live update.

OK, so if you read this far, repo sync your aosp source and build the generic_x86 branch. The binary is aosp/target/product/generic_x86/system/bin/applypatch_static

edit: then after you've done that, use a script to parse the updater-script to extract the patching commands and then she-bang! I'll post mine a little later- I've got to get some flowers for tonight

 

[bgcngm]

Any luck with that? I'm also searching for a way to apply an OTA update to a ROM using Ubuntu.

 

[gee one]

Any luck with that? I'm also searching for a way to apply an OTA update to a ROM using Ubuntu.

Yeah, it works like a charm. The binary is compiled in the aosp/generic_x86 build. There is no need to flash, you can just pull it out of the system/bin folder after compiling the source. You can probably use an "mmm" command to make just files you want, but I'm not sure which package it's in. Then, I use a script that converts the OTA patch commands in the updater-script to a script file that runs on my linuxbox that calls the binary and patches the ROM under linux/ubuntu.

 

[bgcngm]

Do you mind sharing your applypatch binary? I don't have AOSP source on my Ubuntu installation right now, so if you don't mind sharing I wouldn't need to build it from sources...

 

[gee one]

Do you mind sharing your applypatch binary? I don't have AOSP source on my Ubuntu installation right now, so if you don't mind sharing I wouldn't need to build it from sources...

I attached the applypatch binary- it should work on most x86 linux distros, maybe.

As I stated in my earlier post, the binary is built with the x86 build. I'm not so good with the build system, but I think the code is from the bootable/recovery/applypatch. I included the relevant notices for that code.

In the OTA updater-script, there will be a bunch of applypatch commands- you might have to change the name/permissions of the binary, but that syntax should be what you are looking for.

Enjoy!

 

[bgcngm]

Thank you very much for your time. It seems we are almost the only one trying this. The thing is that I really need to get this done because OTA update is failing for me by running it the usual way.

I've tried your binary and it works under my Ubuntu installation, however I was trying to manually patch just one file and got this:

[email protected]:~/Desktop/system/app$ applypatch_static ./ApplicationsProvider.apk - d9023cd58fd055e1ca3c8f8492b2c36aba923c6a 6184 69bea5d5a62980c611b903de8243d980f30e5fb5:../../patch/system/app/ApplicationsProvider.apk.p 

applying patch to ./ApplicationsProvider.apk
target 6184 bytes; free space 1080520704 bytes; retry 1; enough 1
chown of "./ApplicationsProvider.apk.patch" failed: Operation not permitted

I got that chown error message although ApplicationsProvider.apk.patch was created. Is this the patched file?

EDIT: Nevermind, I forgot to run the command as superuser.

EDIT2: Would it be too much if I ask you also to share the script that converts OTA commands from updater-script into the correct syntax to run on Ubuntu?

 

[gee one]

There is no guarantee that this will work on your machine. It's a work in progress and will probably require some editing. This just creates the scripts that extract the commands from the OTA updater-script.

I haven't used it in a while since ASUS hasn't put out any updates lately, but I would copy the updater-script and rename it delete_source. Then I would edit out everything in delete_source except the delete commands. There might be an issue with a trailing ; or similar character. The script runs on the copy of the updater-script to pull out the applypatch commands and the delete_source to pull out the files that need to be deleted from the rom.

Check the syntax, but it will be something like ./tf_file <head of stock rom> <head of patch source>
If you have the donor stock system folder and the patch folder from the OTA at the same dir level, it would be ./tf_file system patch
The script also assumes that the applypatch binary is in your path as apply_patch. You can adjust as needed.

Lastly, my bash skills are probably pretty weak, so if you see ways to improve this script or add error/path checking, feel free to post improvements, etc. There is probably a really slick way to do this with just one script and not a script that makes three other scripts. The historical reason for this is that I used to transfer the files to my transformer and run the script on there. Running in Ubuntu is much faster!

Good luck!

#!/bin/bash

# extract_source is used to pull files to be patched from source tree
# delete_source is ued to delete unneeded files from the source tree
# tf_file applies the patches to the extracted source files

# don't forget to copy the direct updates in the OTA

# usage is apply_ota
# assumes that updater-script and delete-script are at same directory level
# updater-script is from META-INF folder, delete-script is from updater-script


# add error checking to count number of patches and number of patched files
# parse trailing / in folder names


work_dir=`pwd`
tf_file=$work_dir"/tf_file"
extract_source=$work_dir"/extract_source"
delete_file=$work_dir"/delete_source"
work_file=$work_dir"/work_file"

echo "#!/bin/bash" > $tf_file
sed ':a;N;$!ba;s/\n/\ /g' updater-script  | sed  -e 's/;/\n/g' -e 's/\ \ */\ /g' | grep "apply_patch(" | sed -e 's/"//g' -e 's/,\ p/\:p/g' -e 's/,//g' -e 's/package_extract_file(/$2\//g' -e 's/(/\ $1/g' -e 's/))//g' >> $tf_file

chmod 755 $tf_file

echo "#!/bin/bash" > $extract_source

#sed ':a;N;$!ba;s/\n/\ /g' updater-script  | sed  -e 's/;/\n/g' -e 's/\ \ */\ /g' | grep "apply_patch(" | cut -d \"  -f 2 |sed -e s'/^/cp\ --parents\ $1/g' -e s'/$/\ \$2"\/\"/g' >> $extract_source
sed ':a;N;$!ba;s/\n/\ /g' updater-script  | sed  -e 's/;/\n/g' -e 's/\ \ */\ /g' | grep "apply_patch(" | cut -d \"  -f 2 |sed -e 's/^/cp\ --parents\ $1/g' -e 's/$/\ \$2"\/\"/g' -e 's:/system::g' >> $extract_source

chmod 755 $extract_source

sed -e 's:",:\n:g'  -e 's:\ ::g' delete-script  > $delete_file

sort -r $delete_file > $work_file

echo "#!/bin/bash" > $delete_file

grep / $work_file | grep -v /$ | sed -e 's:"/:rm\ \$1/:g' >> $delete_file
grep /$ $work_file | sed -e 's:"/:rmdir\ \$1/:g' >> $delete_file

chmod 755 $delete_file

exit 0

 

[jawad6233]

IMGDIFF Tool Needed...

Hello Sir, i am looking for a tool to generate IMGDIFF patches on windows... i already have a BSDIFF tool, but coudn't find IMGDIFF... Thank You

 

[gee one]

I have no idea about windows, but the source is here.

https://android.googlesource.com/platform/bootable/recovery/+/master

You could install linux and build it from there. Follow these directions http://source.android.com/source/building.html

There is also lots of great info here http://wiki.cyanogenmod.org/w/Development#Learning_To_Build_CM

Good luck!

 

[jawad6233]

Good luck!

I am very thankful to you for replying me sir... these guides are awesome, but i dont think my slow internet connection and PC with 1GB RAM will be allowing me to download or compile
actully i was making a tool to apply OTA Updates in PC, it is semi functional for now, it is able to read updater script and apply bsdiff patches, but can't handle imgdiff patches and now i am stuck at this point because of no imgdiff port for windows... :crying:

i will be very thankful if you or @bgcngm can compile imgdiff at least for cygwin? else i will have to postpond this project until someone else compile and port imgdiff for win/cygwin

 

[gee one]

Sorry- I'm clueless about cygwin. You could try dual booting to linux to compile from there.

 

[jawad6233]

You could try dual booting to linux to compile from there.

Thanks alot for taking your time to reply me sir now it seems like linux is the only option left
as you are a linux user will it be too much if i ask you to find a right guide to compile ota tools only? cause i don't know much about linux, and it will be difficult for me to find right guide to compile ota tools only...

 

[DeckerSU]

Hello Sir, i am looking for a tool to generate IMGDIFF patches on windows... i already have a BSDIFF tool, but coudn't find IMGDIFF... Thank You

Answer is here - [Tool]imgdiff executable for windows. BTW, anybody have applypatch for Win32?

 

[vampirefo]

Hoping this works for lollipop.