Attend XDA's Second Annual Developer Conference, XDA:DevCon 2014!
5,809,814 Members 43,492 Now Online
XDA Developers Android and Mobile Development Forum

[SCRIPT] Repack multiple APKs with proper compression settings (and other stuff!)

Tip us?
 
omniwolf
Old
(Last edited by omniwolf; 24th January 2012 at 12:26 PM.) Reason: v1.5.1
#1  
omniwolf's Avatar
Senior Member - OP
Thanks Meter 459
Posts: 992
Join Date: Jan 2005
Location: sydney
Post [SCRIPT] Repack multiple APKs with proper compression settings (and other stuff!)

Hi Guys,

I was speaking to baadnwz a couple of weeks ago and i asked him which script he used to properly pack all the APKs in his ROM. He told me he used no script, just repacked the APKs he thought were most relevant.
I figured it couldn't be too hard to write something that took an APK, extracted it somewhere, then repacked it with the proper compression, so i wrote one. baadnwz then challenged me to make it work for multiple APKs, so he can just run it against a folder full of them like a lazy asshole. i went one better, and made a script that you could run against a ROM folder structure, it will search out all APKs, repack them, and copy them back into place. HOT!

because this uses recursion and functions to make it easier to write (and because i'm generally comfortable in it), i have written it in VBScript. So this means it won't work on linux as far as i know, sorry. Feel free to convert it or whatevs

WHAT DOES IT DO??????//
1. finds an APK
2. unpacks it in a folder
3. optimises the PNGs (if you want it to)
4. repacks all files as deflate except a blacklist of filetypes*
5. repacks the blacklisted filetypes* as STORE
6. zipaligns the new APK

7. copies it back to the original location, overwriting the original APK
8. finds the next APK, and repeat from step 2

* the blacklisted file types are easily editable. just find the STOREFileNames variable. by default the filetypes are:

"*.arsc", "*.m10", "*.png", "*.wav", "*.ogg", "*.mp4", "*.jpl", "*.jpeg", "*.gif","*.mp2", "*.mp3", _
"*.aac","*.mpg", "*.mpeg", "*.mid", "*.midi", "*.smf", "*.jet","*.rtttl", "*.imy", "*.xmf", _
"*.m4a","*.m4v", "*.3gp", "*.3gpp", "*.3g2", "*.3gpp2", "*.amr", "*.awb", "*.wma", "*.wmv", _
"*.so", "*.dat", "*.bin"

WHY DO YOU WANT THIS??
If certain filetypes in the APK are compressed as STORE (which means they aren't compressed at all), then Android can use them directly. If they are stored as DEFLATE (like all other file types should be), then Android must first decompress the APK somewhere. This means slower apps, more cache, and it's just not how things are supposed to be done. This script makes sure that all files inside all APKs are stored as they should be, and it will optimise PNG files and zipalign the final APK as it goes.

This script is really meant for ROM chefs to run against their ROMs before releasing them, but if you (a mere user) have a ROM that your chef hasn't run this script on, then you can extract the ROM, run the script on it, then recompress the ROM and flash it.

HOW DOES IT DO IT?????///////
  • It uses the command line version of 7-Zip. Download it from here:
    http://7-zip.org/download.html
  • It uses zipalign.exe from the android SDK. if you don't know where this comes from.. maybe you shouldn't be using this script
  • It uses a cool OptiPNG utility to optimise the PNGs if you want it to. This slows the processing down a lot.. but it's up to you if you wanna do it. You'll need to download the utility from here:
    http://optipng.sourceforge.net/


WHAT DO I HAVE TO DO TO MAKE IT WORK//////?
  1. Grab the script, put it in the same folder as the root of your ROM (the script will repack all APKs in the subfolders from where it's sitting, including the APKs in the folder the script is in).
  2. open up the script, there are 5 variables you MUST verify, most likely you will have to change them.. unless you put things on a D and E drive and your name is Nick..

    tempAPKextract = "E:\Temp\tempAPKExtract"
    tempAPKBuild = "E:\Temp\tempAPKBuild"
    sevenZipLoc = """D:\nick\android\7za\7za.exe"""
    zipAlignLoc = """D:\nick\android\android-sdk-windows\tools\zipalign.exe"""
    optiPNGLoc = """D:\android\7-zipA\ROM\optipng.exe"""


    the first two temp ones can be anywhere.. just make sure at least the sub folders up to "tempAPK..." exist (eg in this case the E:\Temp folder must already be there)

    the next three are the locations of the tools required for the script to work, edit to please!
  3. Run the script. If you have set verboseOutput to true (it's false by default), then I would highly recommend you run the script from the commandline with the command:

    cscript fixAllAPKs.vbs

anyways, hope it's useful to you.

DOWNLOAD

CHANGELOG????/
v1.5.1 - cleaned up some code, added in verboseOutput so you can watch the progress as it's fixing (must run as cscript!)
v1.5 - added OptiPNG to the script, enabled by default. Quite slow. Added some more file types.
v1.4.3 - added more file types to STORE, thanks gtg465x
v1.4.1 - minor code clean up, fixed some comments.
v1.4 - fixed the problem where the script wouldn't fix any APKs in the root folder that the script was in
v1.3 - almost doubled the speed of the script. fixed some naughty bugs
XDA II > i-mate PDA2 > HTC Touch > HTC Diamond > HTC Desire > SGS2 > SGS3 > SGS3 LTE > SGN2 LTE > SGS4 LTE > HTC One M8
Running InsertCoin ROM

Are you a ROM chef? Want to make sure your APKs have the correct compression settings, the internal PNGs are optimised, and the APKs are zip aligned? Check out my script:
[SCRIPT] Repack multiple APKs with proper compression settings
The Following 15 Users Say Thank You to omniwolf For This Useful Post: [ Click to Expand ]
 
gtg465x
Old
(Last edited by gtg465x; 21st June 2011 at 09:23 PM.)
#2  
Recognized Developer
Thanks Meter 2,835
Posts: 3,803
Join Date: Jun 2008
edit: nevermind
 
omniwolf
Old
#3  
omniwolf's Avatar
Senior Member - OP
Thanks Meter 459
Posts: 992
Join Date: Jan 2005
Location: sydney
not from what i've been told. if it's not compressed, the android system doesn't have to explicitly decompress it somewhere when it wants to use it, it can just pick it straight out of the APK.
XDA II > i-mate PDA2 > HTC Touch > HTC Diamond > HTC Desire > SGS2 > SGS3 > SGS3 LTE > SGN2 LTE > SGS4 LTE > HTC One M8
Running InsertCoin ROM

Are you a ROM chef? Want to make sure your APKs have the correct compression settings, the internal PNGs are optimised, and the APKs are zip aligned? Check out my script:
[SCRIPT] Repack multiple APKs with proper compression settings
 
gtg465x
Old
#4  
Recognized Developer
Thanks Meter 2,835
Posts: 3,803
Join Date: Jun 2008
Quote:
Originally Posted by omniwolf View Post
not from what i've been told. if it's not compressed, the android system doesn't have to explicitly decompress it somewhere when it wants to use it, it can just pick it straight out of the APK.
Good man. Here are all of the files blacklisted by the default Android Asset Packaging Tool (aapt) if you want to include them.

{
".jpg", ".jpeg", ".png", ".gif",
".wav", ".mp2", ".mp3", ".ogg", ".aac",
".mpg", ".mpeg", ".mid", ".midi", ".smf", ".jet",
".rtttl", ".imy", ".xmf", ".mp4", ".m4a",
".m4v", ".3gp", ".3gpp", ".3g2", ".3gpp2",
".amr", ".awb", ".wma", ".wmv"
};
 
omniwolf
Old
#5  
omniwolf's Avatar
Senior Member - OP
Thanks Meter 459
Posts: 992
Join Date: Jan 2005
Location: sydney
cool, thanks for the info. i've added the missing ones in and uploaded a new version.
cheers!
XDA II > i-mate PDA2 > HTC Touch > HTC Diamond > HTC Desire > SGS2 > SGS3 > SGS3 LTE > SGN2 LTE > SGS4 LTE > HTC One M8
Running InsertCoin ROM

Are you a ROM chef? Want to make sure your APKs have the correct compression settings, the internal PNGs are optimised, and the APKs are zip aligned? Check out my script:
[SCRIPT] Repack multiple APKs with proper compression settings
 
myn
Old
#6  
myn's Avatar
Senior Recognized Developer
Thanks Meter 4,002
Posts: 2,658
Join Date: Nov 2007

 
DONATE TO ME
Nice work.

Suggestion or maybe a challenge for Ver 2:

Given a definition of what languages the user defines, can you repack including only those languages. This would include both drawables and layout language folders.

Removing languages not only speeds up the rom but reduces the apk size which for Sense 3.0 is becoming challenging for many older (> 1 year) devices.

Thanks for your contributions to XDA.
 
gtg465x
Old
#7  
Recognized Developer
Thanks Meter 2,835
Posts: 3,803
Join Date: Jun 2008
Also need to add .so to the blacklist. Ran into a problem with that the other day.
Current Phones:
Moto X GSM Unlocked

Previous Phones:
HTC One Unlocked, Nexus 4, iPhone 5, Galaxy Nexus, iPhone 4S, HTC One X, Nokia Lumia 800, HTC Titan, Samsung Galaxy S II, Samsung Infuse, LG Thrill, HTC Inspire, Samsung Captivate, HTC Aria, Samsung BlackJack II
 
AdamOutler
Old
(Last edited by AdamOutler; 21st June 2011 at 08:14 PM.)
#8  
AdamOutler's Avatar
Recognized Developer
Thanks Meter 9,623
Posts: 5,196
Join Date: Feb 2011
Location: Louisiana

 
DONATE TO ME
Here's mine.

It synchronizes with our SVN repository and builds the rom. Any commits which were made are automatically turned into a new ClockWorkMod flashable package.

Code:
#! /bin/sh  

#This is the repository Directory, change to match your setup 
Repo="~/TeamKomin/trunk" 
#This is the Final folder, this is where the file will end up
Output="~/Desktop"   

#synchronize SVN repository
svn update "$Repo" 
#remove temp folder if exist
test -e "$Output/tmp/" && rm -Rf "$Output/tmp/" 
#make new temp folder
mkdir "$Output/tmp/" 
#copy repository to temp folder
cp -rf "$Repo/data" "$Output/tmp/" 
cp -rf "$Repo/META-INF" "$Output/tmp/" 
cp -rf "$Repo/system" "$Output/tmp/" 
cp -rf "$Repo/updates" "$Output/tmp/" 

#clear SVN folders to reduce size
find "$Output/tmp/" -name ".svn" -type d -exec rm -rf {} \; 
cd "$Output/tmp" 
#remove old files if there
test -e "$Output/AndromedaRepo.zip" && rm -f "$Output/AndromedaRepo.zip" 

#zip up a new Andromeda3 version
zip -r "$Output"/AndromedaRepo.zip ./* 

#done
I've also got a version which uses the Expect command to log into our SFTP server and update the files.teamkomin.com website... I'm not posting that here though as it contains security sensitive information.

Using this method, all members work on the ROM as though it were on the phone... This script packages it. No need to package or unpackage the entire thing.
Flash with Odin on Windows, Linux and Mac. Use JOdin3, Available in a web browser or offline
Check out my developer pages. Add me to your circles on Google Plus.
Wanna see the longest Linux BASH script ever made? click here.
 
omniwolf
Old
#9  
omniwolf's Avatar
Senior Member - OP
Thanks Meter 459
Posts: 992
Join Date: Jan 2005
Location: sydney
Quote:
Originally Posted by myn View Post
Nice work.

Suggestion or maybe a challenge for Ver 2:

Given a definition of what languages the user defines, can you repack including only those languages. This would include both drawables and layout language folders.

Removing languages not only speeds up the rom but reduces the apk size which for Sense 3.0 is becoming challenging for many older (> 1 year) devices.
ok, sounds like a good idea. i'm not actually a ROM developer, so i don't know the ins and outs of the various files. any hints on which files i exclude for various languages? just having a vague look around i see in framework-res.apk there's some raw-ar, raw-cs, raw-en-GB folders, are these the ones you're saying i can strip out?
this will obviously be a lot easier if i can do it programmatically, do you know the rules?

Quote:
Also need to add .so to the blacklist. Ran into a problem with that the other day.
ok cool, added it in and uploaded 1.4.3. thanks dude.

Quote:
Here's mine.

It synchronizes with our SVN repository and builds the rom. Any commits which were made are automatically turned into a new ClockWorkMod flashable package.

I've also got a version which uses the Expect command to log into our SFTP server and update the files.teamkomin.com website... I'm not posting that here though as it contains security sensitive information.

Using this method, all members work on the ROM as though it were on the phone... This script packages it. No need to package or unpackage the entire thing.
ok cool. does this actually repack and zip align the APKs though? this looks like it just takes care of the overall ROM zip structure, not the APKs within..? maybe your script could call mine.. if there was a vbs parser for linux
XDA II > i-mate PDA2 > HTC Touch > HTC Diamond > HTC Desire > SGS2 > SGS3 > SGS3 LTE > SGN2 LTE > SGS4 LTE > HTC One M8
Running InsertCoin ROM

Are you a ROM chef? Want to make sure your APKs have the correct compression settings, the internal PNGs are optimised, and the APKs are zip aligned? Check out my script:
[SCRIPT] Repack multiple APKs with proper compression settings
 
AdamOutler
Old
#10  
AdamOutler's Avatar
Recognized Developer
Thanks Meter 9,623
Posts: 5,196
Join Date: Feb 2011
Location: Louisiana

 
DONATE TO ME
Quote:
Originally Posted by omniwolf View Post
ok, sounds like a good idea. i'm not actually a ROM developer, so i don't know the ins and outs of the various files. any hints on which files i exclude for various languages? just having a vague look around i see in framework-res.apk there's some raw-ar, raw-cs, raw-en-GB folders, are these the ones you're saying i can strip out?
this will obviously be a lot easier if i can do it programmatically, do you know the rules?


ok cool, added it in and uploaded 1.4.3. thanks dude.


ok cool. does this actually repack and zip align the APKs though? this looks like it just takes care of the overall ROM zip structure, not the APKs within..? maybe your script could call mine.. if there was a vbs parser for linux
We keep the APKs on the server for ease of use. It makes it easier to reference a single file for testing that way.
Flash with Odin on Windows, Linux and Mac. Use JOdin3, Available in a web browser or offline
Check out my developer pages. Add me to your circles on Google Plus.
Wanna see the longest Linux BASH script ever made? click here.

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes