[SCRIPT] Repack multiple APKs with proper compression settings (and other stuff!)

omniwolf · Jun 14, 2011

Hi Guys,

I was speaking to baadnwz a couple of weeks ago and i asked him which script he used to properly pack all the APKs in his ROM. He told me he used no script, just repacked the APKs he thought were most relevant.
I figured it couldn't be too hard to write something that took an APK, extracted it somewhere, then repacked it with the proper compression, so i wrote one. baadnwz then challenged me to make it work for multiple APKs, so he can just run it against a folder full of them like a lazy asshole. i went one better, and made a script that you could run against a ROM folder structure, it will search out all APKs, repack them, and copy them back into place. HOT!

because this uses recursion and functions to make it easier to write (and because i'm generally comfortable in it), i have written it in VBScript. So this means it won't work on linux as far as i know, sorry. Feel free to convert it or whatevs

WHAT DOES IT DO??????//
1. finds an APK
2. unpacks it in a folder
3. optimises the PNGs (if you want it to)
4. repacks all files as deflate except a blacklist of filetypes*
5. repacks the blacklisted filetypes* as STORE
6. zipaligns the new APK
7. copies it back to the original location, overwriting the original APK
8. finds the next APK, and repeat from step 2

* the blacklisted file types are easily editable. just find the STOREFileNames variable. by default the filetypes are:

"*.arsc", "*.m10", "*.png", "*.wav", "*.ogg", "*.mp4", "*.jpl", "*.jpeg", "*.gif","*.mp2", "*.mp3", _
"*.aac","*.mpg", "*.mpeg", "*.mid", "*.midi", "*.smf", "*.jet","*.rtttl", "*.imy", "*.xmf", _
"*.m4a","*.m4v", "*.3gp", "*.3gpp", "*.3g2", "*.3gpp2", "*.amr", "*.awb", "*.wma", "*.wmv", _
"*.so", "*.dat", "*.bin"

WHY DO YOU WANT THIS??
If certain filetypes in the APK are compressed as STORE (which means they aren't compressed at all), then Android can use them directly. If they are stored as DEFLATE (like all other file types should be), then Android must first decompress the APK somewhere. This means slower apps, more cache, and it's just not how things are supposed to be done. This script makes sure that all files inside all APKs are stored as they should be, and it will optimise PNG files and zipalign the final APK as it goes.

This script is really meant for ROM chefs to run against their ROMs before releasing them, but if you (a mere user) have a ROM that your chef hasn't run this script on, then you can extract the ROM, run the script on it, then recompress the ROM and flash it.

HOW DOES IT DO IT?????///////

It uses the command line version of 7-Zip. Download it from here:
http://7-zip.org/download.html
It uses zipalign.exe from the android SDK. if you don't know where this comes from.. maybe you shouldn't be using this script
It uses a cool OptiPNG utility to optimise the PNGs if you want it to. This slows the processing down a lot.. but it's up to you if you wanna do it. You'll need to download the utility from here:
http://optipng.sourceforge.net/

WHAT DO I HAVE TO DO TO MAKE IT WORK//////?

Grab the script, put it in the same folder as the root of your ROM (the script will repack all APKs in the subfolders from where it's sitting, including the APKs in the folder the script is in).
open up the script, there are 5 variables you MUST verify, most likely you will have to change them.. unless you put things on a D and E drive and your name is Nick..

tempAPKextract = "E:\Temp\tempAPKExtract"
tempAPKBuild = "E:\Temp\tempAPKBuild"
sevenZipLoc = """D:\nick\android\7za\7za.exe"""
zipAlignLoc = """D:\nick\android\android-sdk-windows\tools\zipalign.exe"""
optiPNGLoc = """D:\android\7-zipA\ROM\optipng.exe"""

the first two temp ones can be anywhere.. just make sure at least the sub folders up to "tempAPK..." exist (eg in this case the E:\Temp folder must already be there)

the next three are the locations of the tools required for the script to work, edit to please!
Run the script. If you have set verboseOutput to true (it's false by default), then I would highly recommend you run the script from the commandline with the command:

cscript fixAllAPKs.vbs

anyways, hope it's useful to you.

DOWNLOAD

CHANGELOG????/
v1.5.1 - cleaned up some code, added in verboseOutput so you can watch the progress as it's fixing (must run as cscript!)
v1.5 - added OptiPNG to the script, enabled by default. Quite slow. Added some more file types.
v1.4.3 - added more file types to STORE, thanks gtg465x
v1.4.1 - minor code clean up, fixed some comments.
v1.4 - fixed the problem where the script wouldn't fix any APKs in the root folder that the script was in
v1.3 - almost doubled the speed of the script. fixed some naughty bugs

gtg465x · Jun 14, 2011

edit: nevermind

omniwolf · Jun 15, 2011

not from what i've been told. if it's not compressed, the android system doesn't have to explicitly decompress it somewhere when it wants to use it, it can just pick it straight out of the APK.

gtg465x · Jun 15, 2011

omniwolf said:
not from what i've been told. if it's not compressed, the android system doesn't have to explicitly decompress it somewhere when it wants to use it, it can just pick it straight out of the APK.

Good man. Here are all of the files blacklisted by the default Android Asset Packaging Tool (aapt) if you want to include them.

{
".jpg", ".jpeg", ".png", ".gif",
".wav", ".mp2", ".mp3", ".ogg", ".aac",
".mpg", ".mpeg", ".mid", ".midi", ".smf", ".jet",
".rtttl", ".imy", ".xmf", ".mp4", ".m4a",
".m4v", ".3gp", ".3gpp", ".3g2", ".3gpp2",
".amr", ".awb", ".wma", ".wmv"
};

omniwolf · Jun 15, 2011

cool, thanks for the info. i've added the missing ones in and uploaded a new version.
cheers!

myn · Jun 17, 2011

Nice work.

Suggestion or maybe a challenge for Ver 2:

Given a definition of what languages the user defines, can you repack including only those languages. This would include both drawables and layout language folders.

Removing languages not only speeds up the rom but reduces the apk size which for Sense 3.0 is becoming challenging for many older (> 1 year) devices.

Thanks for your contributions to XDA.

gtg465x · Jun 17, 2011

Also need to add .so to the blacklist. Ran into a problem with that the other day.

AdamOutler · Jun 17, 2011

Here's mine.

It synchronizes with our SVN repository and builds the rom. Any commits which were made are automatically turned into a new ClockWorkMod flashable package.

Code:

#! /bin/sh  

#This is the repository Directory, change to match your setup 
Repo="~/TeamKomin/trunk" 
#This is the Final folder, this is where the file will end up
Output="~/Desktop"   

#synchronize SVN repository
svn update "$Repo" 
#remove temp folder if exist
test -e "$Output/tmp/" && rm -Rf "$Output/tmp/" 
#make new temp folder
mkdir "$Output/tmp/" 
#copy repository to temp folder
cp -rf "$Repo/data" "$Output/tmp/" 
cp -rf "$Repo/META-INF" "$Output/tmp/" 
cp -rf "$Repo/system" "$Output/tmp/" 
cp -rf "$Repo/updates" "$Output/tmp/" 

#clear SVN folders to reduce size
find "$Output/tmp/" -name ".svn" -type d -exec rm -rf {} \; 
cd "$Output/tmp" 
#remove old files if there
test -e "$Output/AndromedaRepo.zip" && rm -f "$Output/AndromedaRepo.zip" 

#zip up a new Andromeda3 version
zip -r "$Output"/AndromedaRepo.zip ./* 

#done

I've also got a version which uses the Expect command to log into our SFTP server and update the files.teamkomin.com website... I'm not posting that here though as it contains security sensitive information.

Using this method, all members work on the ROM as though it were on the phone... This script packages it. No need to package or unpackage the entire thing.

omniwolf · Jun 19, 2011

myn said:
Nice work.

Suggestion or maybe a challenge for Ver 2:

Given a definition of what languages the user defines, can you repack including only those languages. This would include both drawables and layout language folders.

Removing languages not only speeds up the rom but reduces the apk size which for Sense 3.0 is becoming challenging for many older (> 1 year) devices.

ok, sounds like a good idea. i'm not actually a ROM developer, so i don't know the ins and outs of the various files. any hints on which files i exclude for various languages? just having a vague look around i see in framework-res.apk there's some raw-ar, raw-cs, raw-en-GB folders, are these the ones you're saying i can strip out?
this will obviously be a lot easier if i can do it programmatically, do you know the rules?

Also need to add .so to the blacklist. Ran into a problem with that the other day.

ok cool, added it in and uploaded 1.4.3. thanks dude.

Here's mine.

It synchronizes with our SVN repository and builds the rom. Any commits which were made are automatically turned into a new ClockWorkMod flashable package.

I've also got a version which uses the Expect command to log into our SFTP server and update the files.teamkomin.com website... I'm not posting that here though as it contains security sensitive information.

Using this method, all members work on the ROM as though it were on the phone... This script packages it. No need to package or unpackage the entire thing.

ok cool. does this actually repack and zip align the APKs though? this looks like it just takes care of the overall ROM zip structure, not the APKs within..? maybe your script could call mine.. if there was a vbs parser for linux

AdamOutler · Jun 19, 2011

omniwolf said:
ok, sounds like a good idea. i'm not actually a ROM developer, so i don't know the ins and outs of the various files. any hints on which files i exclude for various languages? just having a vague look around i see in framework-res.apk there's some raw-ar, raw-cs, raw-en-GB folders, are these the ones you're saying i can strip out?
this will obviously be a lot easier if i can do it programmatically, do you know the rules?

ok cool, added it in and uploaded 1.4.3. thanks dude.

ok cool. does this actually repack and zip align the APKs though? this looks like it just takes care of the overall ROM zip structure, not the APKs within..? maybe your script could call mine.. if there was a vbs parser for linux

We keep the APKs on the server for ease of use. It makes it easier to reference a single file for testing that way.

designgears · Jun 21, 2011

http://www.jsware.net/jsware/vblinux.php5

vbscript on linux with wine!

mike1986. · Jun 26, 2011

Got error on line 118 - WshShell.Run

Anything else shall I edit apart of 4 paths? I have 64-bit OS if that matters.

AdamOutler · Jun 29, 2011

I just realized how much easier this would be on linux...

Code:

#! /bin/bash
OutDir=~/Desktop/newRom
InDir=~/Desktop/myRom
zipAlign=/home/adam/code/android-sdk-linux_x86/tools/zipalign
TempDIr=~/Desktop/TempDir
FIFO=~/Desktop/temp1234
realign="apk dll whatever files"


#set things up
rm -Rf "$OutDir"
mkdir "$OutDir"
rm "$FIFO"
mkfifo "$FIFO"


#Move to dir and start script
cd InDir
#Spawn another process to write to the fifo object
find `pwd`>"$FIFO" &
#read each line in the fifo
while read line
do 
    #get each object extension in $realign variable
    for x in $realign
    do 
        #test if the extension matches the realignment 
        if [ $x = ${line#*.} ]; then
            #make the path in case it does not exist, could be optimized
            touch "$line"
            #meh... this would be the command ish...
            "$zipAlign" "$line" "$temp/$line"
        else 
            mv "$line" "$temp/$line"
        fi
    done
done <$FIFO

#this just removes the redundant filesystem created...
#/home/adam/desktop/out/home/adam/desktop/myrom
rm -Rf "$OutDir"
mkdir "$OutDir"
mv "$/TempDIr/$InDIr" "$OutDir"

Untested, should work with some realignment of the variables at the top.

omniwolf · Jul 9, 2011

mike1986. said:
Got error on line 118 - WshShell.Run

Anything else shall I edit apart of 4 paths? I have 64-bit OS if that matters.

i'd say it's a problem with your

sevenZipLoc

variable, it's the first place it gets used. my OS is 64bit too, so that shouldn't be a problem. what does your sevenZipLoc variable look like?

mike1986. · Jul 9, 2011

omniwolf said:
i'd say it's a problem with your

sevenZipLoc

variable, it's the first place it gets used. my OS is 64bit too, so that shouldn't be a problem. what does your sevenZipLoc variable look like?

sevenZipLoc = """C:\Program Files\7-Zip\7z.exe"""

EDIT:

It seems that I forgot to download "command version"...

williamfold · Jul 9, 2011

Any way to add optimizing PNGs to this script as well?

omniwolf · Jul 11, 2011

williamfold said:
Any way to add optimizing PNGs to this script as well?

show me how to do this and i might be able to add it in..

williamfold · Jul 11, 2011

Heres the link to the opensource OptiPNG program to optimize a PNG

http://optipng.sourceforge.net/

afifth · Jul 11, 2011

I hope it perfectly work

gumbung · Oct 8, 2011

wow thanks bro, that's way i need to learn about it

[SCRIPT] Repack multiple APKs with proper compression settings (and other stuff!)

Senior Member

Inactive Recognized Developer

Senior Member

Inactive Recognized Developer

Senior Member

Retired Senior Recognized Developer

Inactive Recognized Developer

Retired Senior Recognized Developer

Senior Member

Retired Senior Recognized Developer

Inactive Recognized Developer

Senior Member

Retired Senior Recognized Developer

Senior Member

Senior Member

Senior Member

Senior Member

Senior Member

Senior Member

Senior Member

Similar threads

Top Liked Posts