FORUMS
Remove All Ads from XDA

[DEV] Dex, Odex and Dexopt

5,530 posts
Thanks Meter: 2,296
 
By Droidzone, Inactive Recognized Developer on 22nd May 2011, 08:02 PM
Post Reply Email Thread
This thread is to discuss details regarding Dalvik VM, creating odex files (Odexing), Deodexing,

A problem I came across while making my Rom was that while using a Data2SD (Sibere's) partition on ext, clearing the dalvik-cache in recovery would result in a bootloop. The logcat showed that system was searching for the dalvik-cache and looping.

To solve this, a solution would be clear dalvik-cache and recreate an optimized dex file. Or another way that comes to mind is to odex the system altogether. While searching for resources on these, I found that a full collection of resources esp description with scripts was not available.

Some information that I've managed to glean about dexes/odexes so far..

Generic script to odex an apk:
Code:
dexopt-wrapper appname.apk[or jar] appname.odex $BOOTCLASSPATH
where $BOOTCLASSPATH is specified in init.rc

For the HTC Desire running a Froyo Sense build,

Code:
$BOOTCLASSPATH=/system/framework/core.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/com.htc.framework.jar:/system/framework/com.htc.android.pimlib.jar:/system/framework/com.htc.android.easopen.jar:/system/framework/com.scalado.util.ScaladoUtil.jar
Script to odex Framework jars:
Code:
for i in /system/framework/*.jar
do
odex=`echo $i | sed -e 's/.jar/.odex/g'`
echo "dexopt-wrapper $i $odex"
dexopt-wrapper $i $odex
done
Script to odex apks in /system/app and /data/app
Code:
for i in /system/app/*.apk
do
odex=`echo $i | sed -e 's/.apk/.odex/g'`
echo "dexopt-wrapper $i $odex"
dexopt-wrapper $i $odex
done
for i in /data/app/*.apk
do
odex=`echo $i | sed -e 's/.apk/.odex/g'`
echo "dexopt-wrapper $i $odex"
dexopt-wrapper $i $odex
done
A few useful links to more information on Dalvik Optimization:
http://www.netmite.com/android/mydro...cs/dexopt.html

Credits:
  1. dsixda (Android Kitchen author), for his deodexing script (do_deodex) included below
I have a few queries regarding these:
  1. What is the difference between a Dex and Odex file (I know dex is optimized odex, but what actual steps are being done to the odex file to make it a dex, and can we script those steps to execute in recovery?)
  2. At what step in Rom install are the dexes created? If it's init.rc that creates these, can we redo these steps in recovery?
  3. Can anyone provide the code to make a Dex file (Not an odex file)
Attached Files
File Type: zip dexopt-wrapper.zip - [Click for QR Code] (2.5 KB, 2368 views)
File Type: zip do_deodex_file.zip - [Click for QR Code] (3.3 KB, 1164 views)
The Following 3 Users Say Thank You to Droidzone For This Useful Post: [ View ] Gift Droidzone Ad-Free
 
 
22nd May 2011, 10:21 PM |#2  
beidl's Avatar
Senior Member
Flag Purbach
Thanks Meter: 181
 
Donate to Me
More
What the preoptimization process does is basically pre binding methods to the framework, replacing some object methods with available native equivalents and pre computing a hash table for faster class name lookup.
That means that every time a the framework dexes change, all other dex files have to be remade using the new available framework.
If you want to create odex files in recovery, there's an easy way and actually I did this already for the old old CM6 nightlies (before we had custom partition tables).

http://forum.xda-developers.com/show...&postcount=200

I took kalis script, packaged it into a flashable zip file and made the script run on flash.
It'll most likely not work with newer ROMs because of missing framework files specified in the BOOTCLASSPATH variable in the script.
Also something useful to add would be removing the classes.dex file from the package after the optimization finished. There must be a zip binary floating around on the interwebs which allows removal of zip file entries using the '-d' argument.

There's no need to manually create dex files as they are stored inside the apks, just being semi-optimized for the specific platform and architecture on installation.
For everything else there's the deodex stuff.
The Following User Says Thank You to beidl For This Useful Post: [ View ] Gift beidl Ad-Free
23rd May 2011, 04:39 AM |#3  
Droidzone's Avatar
OP Inactive Recognized Developer
Flag Kochi
Thanks Meter: 2,296
 
Donate to Me
More
Quote:
Originally Posted by beidl


There's no need to manually create dex files as they are stored inside the apks, just being semi-optimized for the specific platform and architecture on installation.
For everything else there's the deodex stuff.

Thanks.. I'm wondering if the steps of semi optimization can be performed manually. While removing the dalvik-cache stored on ext, the system seems to fail to reload them.. If the odex can be converted to dex, that would be solved.

For Android market, I've tried just renaming Vending.odex as Vending.dex and it worked. But for most other apps, the optimization seemed more..System seems to do the optimization with the binary dexopt, at runtime. On executing it manually at recovery, it gives the msg that it shouldnt be executed.. However if the binary is opened as text, some strings which show arguments can be seen. So it does take arguments, and can be executed:

Code:
DexOptZ: unable to open zip archive '%s'
DexOptZ: unable to open zip archive '%s'
DexOptZ: zip archive '%s' does not include %s
DexOptZ: zip archive GetEntryInfo failed on %s
DexOptZ: extraction of %s from %s failed

Optimization failed
Wrong number of args for --zip (found %d)
%s '%s'bad zip fdbad cache fd BOOTCLASSPATHDexOptZ: BOOTCLASSPATH not set
DexOptZ: truncated BOOTCLASSPATH to '%s'
Not enough arguments for --dex (found %d)
bad vm buildDexOpt: build rev does not match VM: %d vs %d
bad fdbad offsetbad length bad modWhen bad crc bad flags dexopt--zip--dexUsage: don't use this
So if anyone has the aosp source code, the arguments could be retrieved..

If dexopt cant be executed in this way, can dexopt-wrapper create dexes?
23rd May 2011, 11:35 AM |#4  
beidl's Avatar
Senior Member
Flag Purbach
Thanks Meter: 181
 
Donate to Me
More
There is really no need to manually create dex files from the packages, they're recreated when needed.
The only logical reason I can think of is when the second partition is not mounted when creating the dexes, resulting in a faulty symlink, bind mount or wrong permissions.
23rd May 2011, 11:50 AM |#5  
Droidzone's Avatar
OP Inactive Recognized Developer
Flag Kochi
Thanks Meter: 2,296
 
Donate to Me
More
Quote:
Originally Posted by beidl

There is really no need to manually create dex files from the packages, they're recreated when needed.
The only logical reason I can think of is when the second partition is not mounted when creating the dexes, resulting in a faulty symlink, bind mount or wrong permissions.

Yes, that is the reason.
See the adb logcat after a dalvik-cache wipe (dalvik was on ext4)
Code:
--------- beginning of /dev/log/system

etc...
--------- beginning of /dev/log/main

I/dalvikvm( 2046): Unable to open or create cache for /system/framework/com.scalado.util.ScaladoUtil.jar (/data/dalvik-cache/[email protected]@[email protected])

D/dalvikvm( 2046): Unable to process classpath element '/system/framework/com.scalado.util.ScaladoUtil.jar'

E/dalvikvm( 2046): ERROR: no valid entries found in bootclasspath '/system/framework/core.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/com.htc.framework.jar:/system/framework/com.htc.android.pimlib.jar:/system/framework/com.htc.android.easopen.jar:/system/framework/com.scalado.util.ScaladoUtil.jar'


E/keystore( 2054): chdir: /data/misc/keystore: No such file or directory

D/AndroidRuntime( 2053):
Ext4 was to be mounted by the busybox runparts procedure..
However it seems that when there is a missing dalvik-cache, the bootloop is before the busybox runparts is invoked?
23rd May 2011, 12:27 PM |#6  
beidl's Avatar
Senior Member
Flag Purbach
Thanks Meter: 181
 
Donate to Me
More
Check if the runparts invokation is correct in the init.rc script on the ramdisk (correct order).
Also keep in mind that ext4 depends on external kernel modules on non-AOSP kernels, when the dalvik-cache wants to be created before the module has been loaded, then there is a logical reason for this. And without the kernel module there is no mounting of ext4 partitions (again, Sense kernels only).
23rd May 2011, 12:36 PM |#7  
Droidzone's Avatar
OP Inactive Recognized Developer
Flag Kochi
Thanks Meter: 2,296
 
Donate to Me
More
Quote:
Originally Posted by beidl

Check if the runparts invokation is correct in the init.rc script on the ramdisk (correct order).
Also keep in mind that ext4 depends on external kernel modules on non-AOSP kernels, when the dalvik-cache wants to be created before the module has been loaded, then there is a logical reason for this. And without the kernel module there is no mounting of ext4 partitions (again, Sense kernels only).

The order seems to be correct..(My init.rc)
The kernel is a custom kernel based on the original Sense kernel, but with internal ext4 support added (no external ext4 module needed)
I dont see a dexopt command in init.rc however, and was wondering where this command kicks in..
23rd May 2011, 02:41 PM |#8  
beidl's Avatar
Senior Member
Flag Purbach
Thanks Meter: 181
 
Donate to Me
More
The checks and dex optimization/creation start with the launch of the zygote process which in turn should start the dalvik vm.
25th May 2011, 04:35 PM |#9  
Senior Member
Flag Suzhou
Thanks Meter: 7
 
More
great tool, thanks.
29th May 2011, 05:47 PM |#10  
Droidzone's Avatar
OP Inactive Recognized Developer
Flag Kochi
Thanks Meter: 2,296
 
Donate to Me
More
Quote:
Originally Posted by beidl

The checks and dex optimization/creation start with the launch of the zygote process which in turn should start the dalvik vm.

Is there a way to force it to run at boot? If app dexes are missing, system re creates them. But not so for system dexes

Sent from my HTC Desire using XDA Premium App
30th May 2011, 12:34 PM |#11  
beidl's Avatar
Senior Member
Flag Purbach
Thanks Meter: 181
 
Donate to Me
More
Quote:
Originally Posted by droidzone

Is there a way to force it to run at boot? If app dexes are missing, system re creates them. But not so for system dexes

Sent from my HTC Desire using XDA Premium App

It recreates EVERY missing dex file. If it does not, there must be something different than the dalvik vm be totally ****ed up.
Post Reply Subscribe to Thread

Tags
dex, dexopt, odex

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes