[01 DEC 09] TUTORIAL: How I cook ROMs for XPERIA + all my kitchens, tips and tricks

.


PS1 My ROMs are NOT intentionally protected: feel free to dump and use any components you please
PS2 Don't be shy: ASK !! but don't be dumb: SEARCH !!

INTRODUCTION​

• Glad to see everybody is trying to cook these days: we are all here to learn and share
  Following xda-devs spirit, I am writting this little guide to help those who could be intrested in cooking their own ROMs
  Don't be shy: If I did it, YES YOU CAN do it too !!

• Let's get real: NO COOKED ROM WILL 100% FULLFILL YOUR NEEDS/EXPECTATIONS BECAUSE WE ARE ALL DIFFERENT
  Let's get real again: THE ONLY WAY TO STOP COMPLAINTING ABOUT "this" OR "that", FLAMING A CHEFF'S TREAD, IS BY DISCOVERING HOW HARD AND COMPLICATED THE PROCESS IS, but don't dismay: this thread is meant to guide you step by step into the cooking world

• I have been reading and learning a lot, trying to build something that would suite MY OWN NEEDS, I have been using my ROMs for quite a while and I am happy and satisfied: I show off my phone and I can proudly say "I did it myself"
  So I invite all of you to give it a try too: it's worth the experience

But first, may I invite you to take a look at some works:

• Link to latest SEX1 R5 -release 5- ROM thread (kitchen and complete cooking log at post 3)
  
• Link to DEUTSCH SEX1 ROM + KUCHEN thread
  
• Link to LATIN SEX1 ROM + COCINA thread
  
• Link to SEX1 R4 public beta testing thread
  
• Link to SEX1 R3 -release 3- PURE, BareBone & XPERT ROMs thread
  
• Link to SEX1 R3 public beta testing thread
  
• Link to SEX1 release 1a ROM thread

STEP 1: BASIC REGISTRY SKILLS​

Every good cooker/cheff needs basic REGISTRY skills
As a good reading/searching-needed noob, I have compiled some knowledge and here are my 2 cents:

• BASICS

1- Registry keys can be generated (created) or changed with a RGU file (unicode format) or XML/PROVXML file (UTF-8 format)
Both can be edited with notepad, but make sure you choose the right format when saving
(use "save as": it will let you choose format)
Differences:
- UTF-8 for XML or unicode for RGU
- dword values must be hexadecimal in RGUs and decimal in XMLs
- RGUs store registry keys only, XMLs store registry keys, file operations, etc

2- A PROVXML file is a glorified XML: it loads automatically when cold-booting the device (hard reset or first boot), which means all of it's instructions, file operations & registry keys will be auto-executed and will override similar instruction from RGUs
In the other hand, XMLs have to be invoked ("runned") during customization (usually from OEM\Operatorpkg\config*.txt)

• ORDER OF IMPORTANCE

1- When cooking the ROM, all RGU files are compiled and gathered to create the main, barebone system registry (see Jack's comment some posts down)

2- After flashing the phone, when the ROM is loaded for the first time (when it takes forever to boot = "cold boot", same as after a hard-reset), all commands inside all PROVXMLs are auto-executed, overriding corresponding registry keys from REG/RGU files

3- After the "set date" and "align" screen, the auto-customization process starts:
Anything inside "config.txt" will be autoexecuted (apply themes, install CABs, run XMLs...)
Here is where I execute my customization XMLs and any required OEM customization XML
Here is an example "config.txt":


- As a side note, "config.txt" usually goes inside OEM\OperatorPkg, and runcc.lua governs which files are auto-launched
- I put most of the big CABs inside OEM\PACKAGES, eg. PACKAGE_Kovsky_ActiveX_Deployment
- I put the small or orphan CABs into OEM\OperatorPkg, together with my XML files, to keep them handy and keep them in mind at a glance towards the customization files

Conclusion: Better put all of your important registry tweaks in XML or PROVXML files instead of searching for a specific registry key inside more than one hundred RGUs, anyway they will be overrided by any XML or PROVXML instruction pointing to that same registry key

• COOKING IMPLICATIONS

1- If required, each OEM or SYS PACKAGE has its own RGU file and/or PROVXML files
Some PACKAGES don't need registry settings, so they don't have an RGU inside

2- The RGU name must be THE SAME as the PACKAGE identifier DSM file

3- Only ONE RGU file is allowed per PACKAGE, but each PACKAGE can contain as many PROVXMLs as required

4- PROVXMLs are a very useful tool because we can add lots of customized instructions to our ROMs ("move this file", "delete that file", "rename this one", "remove Hidden attributes", etc)

5- I prefer to edit the "stock" PROVXMLs instead of creating new ones: a single format error will make the whole XML unusable and the customization process will omit it

• SAMPLE RGU FILE

BASIC RGU / REG RULES:
- First line must say "REGEDIT 4" (don't ask me why )
- Anything apres a ";" is just a comment and won't be processed
- Last line must be empty (again don't ask me why )


Again: DON'T FORGET to leave the last line empty !!!

• XML / PROVXML SAMPLE

Check this *.PROVXML file guys, I edited it to rename my SEX1_stwater*.png custom backgrounds to stwater*.png (thus making them the "default" backgrounds after the flash)
Also, there's a registry change example included in this sample XML:




Continued...

STEP 2: THE KITCHEN​

Improvements: 1- During cooking process, it will let you choose between LZX or XPR compression without having to change any files (now it uses th "mythical" universal cecompr.dll ) 2- Before closing, it pauses to let you explore or copy temp files 3- Smaller in size and super FAST: cook a 100 MB ROM in one minute (using XPR)

[01 JUN 09] SEX1 KITCHEN, more simple, small & easy to use. Download at post footer
[01 JUN 09] COCINA SEX1 con ejecutable ('script') en español, más sencilla, pequeña y fácil de usar. Descarga al pie de este post

• BASICS

Every Kitchen requires Microsoft Visual Basic redistributable & Microsoft .NET Frameworks 3.5 to run
Download here: http://www.microsoft.com/downloads/details.aspx?FamilyID=9b2da534-3e03-4391-8a4d-074b9f2bc1bf&DisplayLang=en

Some Kitchens must to be placed in the root directory C:\
The SEX1 kitchen can be placed anywhere in your PC, even in your desktop if you wish

• INTRODUCTION

A "kitchen" is a compilation of tools, some are optional and some required to cook a ROM
The SEX1 Kitchen has the minnimum required tools only, making it smaller

Some users have written SCRIPTS to automate the cooking process and make things easier, so even I can cook
The script guides you instead of manually launching every required app in the correct order
Just click "Cook.bat" "kovskykitchen.bat" or "buildNB.bat" (it depens on which kitchen you are using) and the automated script will do the rest

This SEX1 Kitchen is a modified derivant from Bepe's old Kitchen structure with updated tools, and in this case I wrote the script for the automated process myself

Erviu's Visual kitchen is a more modern, UI-oriented alternative, but by the time being I will focus on the structure and some basic concepts for this classic ole' style kitchen from Bepe because Visual Kitchen uses a dfferent folder structure

• STRUCTURE

Inside the Kitchen you will find 4 folders and the Kitchen's batch script:

BATCH SCRIPT:

- When finished populating OEM, SYS and ROM (XIP) folders with your desired OEM packages and CE|OS build choice, double click the batch script to start cooking and have a smoke or a coffee while it finishes (this Kitchen is unattended, it takes around 2 minutes to auto-cook a 100 MB ROM with standard compression)
For the english Kitchen: double click "Cook.bat"
Para la Cocina en español: doble click "Cocinar.bat"

- The batch script can be edited with notepad
I inserted some REMARKS ("REM") inside the script:
1- First one if you want to include a pause in the right place to manually edit file attributes before cooking the ROM
2- Another one in case you want to Hex-edit "imgfs.bin" to manually change compression from stock WRX to new LZX

NOTICE 1: LZX compression can be used in this Kitchen by populating the \ROM\ folder with a LZX compression enabled "os.nb.payload" or by editing the new "imgfs.bin" during the cooking process

NOTICE 2: LZX uses a different compressor DLL, it can be downloaded at post footer and relaced in \tools\ folder

KITCHEN FOLDERS:

- The \Tools\ folder is the "oven": it contains all the required tools to cook the ROM
I added step by step explanations into the script, so you can check it to see what happens and which apps are called, to better understand the cooking process: It's worth taking a look
We should be grateful with bepe, bot, ervius, MrVanx, OM-BY, blasphemer, udk and many others for these amazing tools
One year ago the process was SO TEDIOUS and HARD... now it is a matter of clicks !!

- The other 3 folders, OEM SYS & ROM, are where the future ROM parts must be placed:
Download the Kitchen first, then populate these folders with your desired recipee

- Where to get OEM, SYS and ROM (XIP) contents from ??
Use mines (from this tutorial), or dumped ones, or from other users who have shared them
There are many many available around the subforum, just do your homework and search

NOTICE 1: SYS packages must be 192 dpi 480 x 800 resolution !!

NOTICE 2: ROM (XIP) with os.nb.payload must come from a XPERIA, otherwise it must have been ported for the XPERIA

NOTICE 3: If you plan to use a ROM (XIP) folder from jerpelea or my newer "SEX1 release 3", the proper compressor DLL must be downloaded and replaced at "/tools/" folder (the ROM compression selection lyes inside "os.nb.payload")

After populating these folders, your future ROM is basically already there, dismantled into the 3 folders:
- ROM folder is the core
CE build version is here, in "xip.bin" (in some kitchens like this one, "xip.bin" is already inside "os.nb.payload")
- SYS folder contains all Microsoft Operatin System shizzz (media player, Titanium, bootscreen, backgrounds, themes, calendar... etc)
OS build version is here, in \OS\ folder
- OEM folder contains Sony or HTC customizations (Dialer skin, Xperia Panels, TouchFlo...)
OEM version (customization) is here, in \OEMVersion\ and \OperatorPkg\ folders

- In most kitchens your new cooked ROM will "appear" in the RUU folder, which contains a "Rom Update Utility" app required to flash your fresh new custom smokin' cooked ROM to the phone
In the SEX1 Kitchen the new cooked ROM will "appear" in the root kitchen folder, renamed to kovsimg.nbh, ready to be flashed with the microSD method (safer IMHO)
Of course it can be renamed to RUU_signd.nbh and USB-flashed from the PC with the RUU app too

• OPTIONAL TOOLS FOR THE SEX1 KITCHEN

Along with the Kitchen, I have uploaded some very useful tools to this post
They are not required but some people could find them usefull and can be optionaly added to the kitchen:
(I wrote and included instructions for each tool inside the ZIP)

1. xvi-32 Hex Editor -> useful to hex-edit files (I Hex-edit my ROMs to change the ROM initial date, tutorial @ post 90)
2. Page Pool changer by OM-BY -> Changes the ROM page pool size (some say 0 is better, others prefer 8MB or even 16MB. I use 0=auto)
3. SYS version by MRVanx -> Gives the real version of every package in SYS folder (don't be surprised: all SYSs floating around are hybrids somehow)
4. RUU ROM Update Utility 3.22.4.3 for Xperia X1 -> To flash ROMs from the PC
5. LZX compression: To make the SEX1 Kitchen COMPATIBLE with LZX-compressed ROM/XIPs, just replace the CE compressor DLL in \TOOLS\ folder with the one posted below

Continued...

STEP 3: The OEM folder, HERE'S WHERE WE'RE GONNA PLAY !!​

• Downloads

Download OEM, ROM & SYS folders (OEM base: R3AA010 WWE) (XIP|SYS 21815) here from SEX1 R3.61 XPERT [01 JUL 09]

Download OEM folder (base: R2A+R3A "beta" WWE) here from W4X.23 [01 MAY 09]



NOTICE: All OEM folders and PACKAGEs I use are optimized for Bepe's old style Kitchen

• Change Log of my OEM packages

release 3 OEM change log:

I dumped R3AA010 UK ROM -> deleted unnecessary OEM packages and streamlined others into \OperatorPkg ("AAinfo.txt", "runcc.lua", operators config "Provision.zip", etc), but kept \ExtendedPkg,
then I:

- Edited OEM\OEMDrivers RGU to:
enable XPERIA start up anime from cold boot (keys were originally added during customizaton in "fileoperations.xml"),
disable SMS sent notification pop up ("NoSMSsentmsg" value 0 -> 1)
disable reg keys for Auto Error Reporting's fatalerror DLL

- Edited OEM\ExtendedPkg\"fileoperations.xml" to:
remove the Bond trailer link inside

- Deleted "fatalerror.dll" in OEM\OEMDrivers
- Deleted "Bond trailer.mp4" and Handango store link in OEM\ExtendedPkg

- Added PACKAGE SD tune UP by tnyynt
- Added my multi-PACKAGE MS SDK certificates + registry lock OFF + CAB insallation warnings disabled

W4X.23 OEM change log:

I dumped XPERIA R2A ROM & downloaded R3A beta OEM packages @ cheffs develop and share centre thread,
then I:

- Deleted unnecessary OEM folders, gathered all XPERIA SEMC contents (from \ExtendedPkg) into two CABs, put them inside \OperatorPkg and edited "config*.txt" to invoke that CABs during customization

- Wrote XMLs for SEMC file operations and edited "config*.txt" to invoke those XMLs during customization (after my SEMC CABs installations, otherwise files wouldn't be found !!)


- Deleted Bejeweled, Sudoku and other unused packages
- Replaced R2A "kovsky" packages with R3A beta packages
- Added PACKAGE for Xtrakt game
- Added PACKAGE for other Panels (Growing panel, Facebook panel, etc)

- Edited OEM\OperatorPkg\"panelconfig.xml" accordingly to the new panels I added, and in my desired panel slot order

• OEM - PACKAGES TUTORIAL

Inside SYS and OEM folders we will find PACKAGES
- A PACKAGE contains an app or util or configuration or else
- A PACKAGE is a group of files that do something: There's a PACKAGE for Adobe reader, another one for OEMDrivers, Dialer, VideoTelephony, GoogleMaps, etc
- Basically speaking, this is what we call "cook": to REMOVE or ADD PACKAGES to a recipe and customize the ROM with apps and utils or tweaks

MORE EASYER ?? Bepe's ole style kitchen uses bepe's BuilOS 0.91 utility to control which PACKAGES are included or not in the new ROM, without having to delete PACKAGES folders every time
You need to create a "option.xml" file inside each PACKAGE for the BuildOS to display it in the list of available, selectable PACKAGES
If a PACKAGE doesn't contain "option.xml" BuildOS will assume it must be cooked, by default
How to create "option.xml" ?? It can be manually created, or automatically created with ervius' package creator

HOW ARE FILES ARANGED ??

- Most files in a ROM (inside OEM or SYS PACKAGES folders) are "normal" files
- But some special "system" files have been disasembled in "Modules"
Eg.: If you search for "shellres.dll" you will not find it, but you will find a folder called "shellres.dll" with *.bin, *.txt and some S000, S001, S002 etc files inside: This is "shellres.dll", but disasembled into "modules"

"recmod.exe" converts Files <-> modules
Dissasembled files are re-assembled when the ROM is flashed


How to tell a file not to go to \Windows ??
Read fards' explanation on post 12 from this thread Initflashes.dat governs what folders are created and which files are moved
Another way to move files is using ye ole *.PROVXML files with proper instructions (I suggest you to open and study some of the stock *.PROVXML files to understand the basic syntax, it's very easy once you have seen a couple of them)

SELECTING ROM CONTENTS

Each package is an app or utility or program or set of customizations, you can add or remove at will, replace a "KOVSKY" app for a "RHODIUM" or "BLACKSTONE" one, but beware: some apps require others !!! Example: If I delete "PACKAGE_Kovsky_PhoneCanvas_VT" (which i the XPERIA phone dialer) and replace it for another dialer skin, I should also delete (or replace) "PACKAGE_Kovsky_VideoTelephony" (It wouldn't make sense to mix up the skins)


CUSTOMIZING / TWEAKING THE ROM

Most of the more important user-related registry keys are inside OEM folder packages
As I stated before, registry changes can be made "inside" every RGU or, as Itje suggested, better put 'em on a PROVXML file, so the "basic" structure of the ROM is not altered next time you want to cook a different ROM and you don't want that change again

TIPS

- Build your mental "structure" of what you want in your ROM

- Start with basic packages cook the ROM and flash it, if it boots and workss, continue tweakin' / moddin'

- Make "maybe yes/maybe not" apps or utils (packages) OPTIONAL in BuildOS with Ervius Package Creator
(The file "option.xml" will not be cooked the ROM, it is there ONLY to tell the kitchen "this is an optional package and the cheff is going to decide wether to select it or not while cooking")

- Avoid LOTS of wasted time by following this advice from a somnolent beginner: KEEP TRACK OF EVERYTHING YOU DO -> WRITE AND MAINTAIN A CHANGE LOG !!!
Who knows ?? Maybe next day you won't remember "which" RGU you edited late last night, or even worst: "what da heck" you tweaked, or "what for"

- Some programs can not be added as packages because some of their files may have repeated names with other packages:
THE KITCHEN WILL STOP COOKING AND WARN FOR A "REPEATED" FILE
(the ROM can't flash 2 files with the same name to \Windows - remember everything goes into \Windows folder during first boot !!)
CABs are very usefull in this case, but beware:
CABs WILL NOT "auto-install", you have to "invoke" them during the CUSTOMIZATION PROCESS, usually adding a line into a "config*.txt"
I chose to drop some CABs inside "OperatoPkg" (again: to keep an eye on them), and kept "stock" PACKAGES for other CABs (check Xtrakt or Opera packages as an example, and open the config*.txt files to see how these CABs are "invoked" to install during first boot customization process)

SOME PACKAGES

• Attached to this post footer you will find a OEM PACKAGE I prepared, meant to add the XPERIA boot start up animation to any ROM (my OEMs all have the animation enabled by default)
  
• BatteEnhancer will include Ethermind's 1% battery driver for XPERIA with required registry changes
  
• SDK+CAB-REG will install SDK certificates (SDKcerts), disable the "non-signed" warnings when installing CABs and turn registry security off (to be edited with any reg editor)
  
• SD tune UP from tnyynt will make your SD card read/write faster
  
• .NET Frameworks (.NET CF) version 3.7

Continued...

STEP 4: THE XIP AND SYS FOLDERS FROM MY ROMS​

• Downloads

Download ROM (XIP) folder OS build 21806 here from SEX1 release 3

Download SYS folder CE build 21806 here from SEX1 release 3

Download ROM (XIP) folder OS build 21500 here from W4X.23 (source: jerpelea's kitchen)

Download SYS folder CE build 21500 here from W4X.23

• Some changes I often make inside SYS folder

- SYS\Base_DPI_192\: Customize "StartMenu_Selection.png" (honeycomb background for selected item)
- SYS\Base_dpi_192_resh_480_resv_800\: Customize "stwater*.jpg" (XPERIA silver theme aluminum background)
- SYS\Redist_dpi_192_resh_480_resv_800\: Customize "welcomehead.192.png" (WM6.5 orange bootscreen says "SEX1")

Applying Halsted's principles to the cooking process, better avoid making changes inside the SYS folder,
instead, now I run my personalizations during the customization process (remember "\OEM\OperatorPkg\config*.txt" ?? Yey !! there, using some XMLs and CABs !!)




- XIP (CE build version) and SYS (OS build version) can be upgraded separatedly, which means you can have a "21500|21806" build or whatever
- Language and resolution can be changed without having to port



To upgrade SYS build version:

1. Replace folders inside SYS with newer build ones

Done

- Beware !! don't change ".VM" and "WinCENLS_WWE" folders, they are device specific
- The Kitchen's automated "G'reloc" will do the rest of the dirty work during cooking (reallocating modules in ROM memory)
- Wanna find out what SYS version each folder is ?? Use SYS version by MRVanx



To upgrade XIP build version:

1. Insert the new "xip.bin" into "os.nb.payload"

Easy, ain't ??

- Beware !! "xip.bin" is device specific
- If a "xip.bin" containing the desired OS build is not available for the device, it can be ported from any "xip.bin" (from other HTC device), then it can be inserted into the native "os.nb.payload"
- MORE INFO on how to extract "xip.bin" from "os.nb.payload", how to port "xip.bin" and how to insert the new "xip.bin" into payload: go to page 11, post 106



To change ROM resolution:
Let's say you just put your hands on a newer SYS build, but it is QVGA (XPERIA is WVGA):

1. All SYS folders labeled "resh_240_resv_320" should be replaced with "resh_480_resv_800" WVGA resolution folders

That's it !!

- TIP: It doesn't matter if correct res folders are taken from a previous build
- Syntax Note: "resolution horizontal 240 x resolution vertical 320"... get it ??



To change ROM language:
Let's say you want to convert the ROM to spanish language:

1. Dump a Spanish OEM and add folders to OEM (at least the basic OEM ones)
2. Replace all "Lang_0409" (WWEnglish) folders in SYS with "Lang_0c0a" (spanish) folders
3. Edit ROM\XIP\ RGU: change MUI locale lang from 0409 to 0c0a

Piece of cake

- Beware !! ".VM" and "WinCENLS_WWE" folders are device & language specific
- TIP: When editing the RGU, search strings for "lang" or "0409" & you will know what to change






KEEP READING THE THREAD GUYS, you can find more useful info and tips from Xperianced senior cookers !!

.

Thanks for pointing out jack

Your help and tips are greatly appreciated, as much as your ROM works

.

kluks said:

great tutorial. thanks again.

I have downloaded the OEM package and tried. Everything works.

I have question:

Do you know how to turn off the notification stuff on the title bar? Every time when I press the task manager icon, I either hit the battery or the speaker volume icon, therefore these notifications pops up. I want to make the battery and speaker icon non-touchable.

Click to expand...

Click to collapse

I know how to disable the battery icon click -> power app in WM6.5:
undo the "click battery opens power control panel" REG trick

(batery icon tap response comes disabled by default... waiting for the NEW POWERDRIVER surprise )

@FARDS:

MANY TAHNKS: GOOD POINT,
Will quote your text in front page

XIP and SYS folders uploaded

SYS changes explained

more details added to Step 1 and 2