[Tool] SparseConverter v1.0.1

[JulesJam]

I think I figured it out. thanks!

FYI - for anyone who wants a step-by-step on how to use a hex editor on a motorola image to remove the header and the footer:

1) Download the hex editor here:

http://mh-nexus.de/en/hxd/


2) Click Edit-> Select Block and in the pop-up select dec at the bottom. Then put 0 in the Start-offset and 131072 in the length. Click OK to select the header. Then click Edit-> Delete to delete the header.

3) At the top where it says hex, change it to dec using the pull down. Click Search-> GoTo and in the pop-up select dec and end (backwards) and in the offset put 4096. The blinking cursor will be on the last pair of the row ABOVE the one you want. Write down the number of the row right below where the blinking cursor is.

4) Click Edit -> Select Block and in the pop-up select dec, in Start offset put the row number from step #3 and in Length type in 4096. Then click OK then click Edit-> Delete to delete the footer.

5) Save the file.

 

[Fedex15]

FYI - for anyone who wants a step-by-step on how to use a hex editor on a motorola image to remove the header and the footer:

1) Download the hex editor here:

http://mh-nexus.de/en/hxd/


2) Click Edit-> Select Block and in the pop-up select dec at the bottom. Then put 0 in the Start-offset and 131072 in the length. Click OK to select the header. Then click Edit-> Delete to delete the header.

3) At the top where it says hex, change it to dec using the pull down. Click Search-> GoTo and in the pop-up select dec and end (backwards) and in the offset put 4096. The blinking cursor will be on the last pair of the row ABOVE the one you want. Write down the number of the row right below where the blinking cursor is.

4) Select Block and in the pop-up select dec, in Start offset put the row number from step #3 and in Length type in 4096. Then click OK then click Edit-> Delete to delete the footer.

5) Save the file.

Hello, I have motorola xt1068 I drew system_motog.img to edit and convert deodex rom. Deodex entire rom (app, framework, priv-app) and converted the system folder system.img with this command "make_ext4fs -s -l 1024M system_new.img system_mnt". Then when the attempt by fastboot flash fails because files have to be 256MB max.I try using the sparceconverter and pulls me error "Image size is not a multiple of 4096 bytes". I try to follow his tutorial HEX EDIT but I still the same, probe "sparseconvert / compress 256MB system_motog.img system" and the command works well, but not with the new image.Where you can be the error. When I follow the tutorial do not understand a part (my language is Spanish), when "The blinking cursor will be on the last pair of the row ABOVE the one you want says. Write down the number of the row right below Where the blinking cursor is . ", what would the code / line, I could put a picture, I will leave one to have if it is correct. Thank You

 

[JulesJam]

Hello, I have motorola xt1068 I drew system_motog.img to edit and convert deodex rom. Deodex entire rom (app, framework, priv-app) and converted the system folder system.img with this command "make_ext4fs -s -l 1024M system_new.img system_mnt". Then when the attempt by fastboot flash fails because files have to be 256MB max

Did you try using mfastboot to flash it?

http://forum.xda-developers.com/attachment.php?attachmentid=2427667


mfastboot can flash the single file system.img. That may be all you need to do. Please let me know if it works.

 

[Fedex15]

Did you try using mfastboot to flash it?

http://forum.xda-developers.com/attachment.php?attachmentid=2427667


mfastboot can flash the single file system.img. That may be all you need to do. Please let me know if it works.

Yes, i try with fastboot and mfastboot, and its the same. I put the folder (framework, app, priv-app) in one zip and flash by recovery. Thanks

 

[JulesJam]

I try to follow his tutorial HEX EDIT but I still the same, probe "sparseconvert / compress 256MB system_motog.img system" and the command works well, but not with the new image. Where you can be the error. When I follow the tutorial do not understand a part (my language is Spanish), when "The blinking cursor will be on the last pair of the row ABOVE the one you want says. Write down the number of the row right below Where the blinking cursor is . ", what would the code / line, I could put a picture, I will leave one to have if it is correct. Thank You

Is this the step you don't understand?

3) At the top where it says hex, change it to dec using the pull down. Click Search-> GoTo and in the pop-up select dec and end (backwards) and in the offset put 4096. The blinking cursor will be on the last pair of the row ABOVE the one you want. Write down the number of the row right below where the blinking cursor is.

 

[fenixus]

yes,
fastboot flash system system.img_sparsechunk1
fastboot flash system system.img_sparsechunk2
fastboot flash system system.img_sparsechunk3

make sure the system partition size match the image that you're trying to flash, to flash a bigger image you'll have to reflash the partition table and all the partitions (boot, recovery etc.)
p.s. not sure what "not working" means, what error do you get exactly?

I don't understand how that set of commands might work if each consequtive fastboot flash system command starts with erasing system partition (and whatever chunk was uploaded before), so in the end user ends up with empty partition and only the last chunk uploaded. I think there was a fastboot switch -u to prohibit erasing partition before formatting, not sure if it works for flashing like that.

Great progress! In my case
I generate my system.img.raw by
adb root
adb wait-for-device
adb shell "dd if=/dev/block/platform/f9824900.sdhci/by-name/system of=/sdcard/system.img.raw"
adb pull -p /sdcard/system.img.raw

1st Trail below will NOT works, but at the end you will see my solution:
Windows cmd log:
E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>sparseconverter /compress system.img.raw .\ 536870912
...
pause
E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_
sparsechunk1
target reported max download size of 536870912 bytes
erasing 'system'...
OKAY [ 0.591s]
sending 'system' (524284 KB)...
OKAY [ 1.350s]
writing 'system'...
OKAY [ 13.953s]
finished. total time: 15.897s

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_
sparsechunk2
target reported max download size of 536870912 bytes
erasing 'system'...
OKAY [ 0.532s]
sending 'system' (524287 KB)...
OKAY [ 1.351s]
writing 'system'...
OKAY [ 8.678s]
finished. total time: 10.565s

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_
sparsechunk3
target reported max download size of 536870912 bytes
erasing 'system'...
OKAY [ 0.524s]
sending 'system' (411515 KB)...
OKAY [ 1.075s]
writing 'system'...
OKAY [123.069s]
finished. total time: 124.671s

Please note even we pass the fastboot flashing, but it does NOT works after flash is finished, the phone can reboot to Linux logo, but block there, can not start Android Animation and go to desktop.
E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>adb shell
- exec '/system/bin/sh' failed: No such file or directory (2) -

SO, SOLUTION is:
as an conclusion, it seems fastboot can not manage the multi input images well, BUT it can handle a single compressed image files, so set the max image side to the partition size to 2684354560 can make fastboot works well.
I need to find my original dd output file system.img.raw in 2.5GB, then:
Windows cmd logs:

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>sparseconverter /compress system.img.raw .\ 2684354560

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_sp
arsechunk1
target reported max download size of 536870912 bytes
erasing 'system'...
OKAY [ 0.767s]
sending sparse 'system' (521371 KB)...
OKAY [ 4.530s]
writing 'system'...
OKAY [ 9.577s]
sending sparse 'system' (511116 KB)...
OKAY [ 6.465s]
writing 'system'...
OKAY [ 10.833s]
sending sparse 'system' (445892 KB)...
OKAY [ 6.274s]
writing 'system'...
OKAY [139.377s]
finished. total time: 177.831s

that will bring my system.img.raw back to my phone WELL
you can see my rooted su is there, ^_^:
E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>adb shell
[email protected]:/ $ su
su
[email protected]:/ #

thanks for the idea!
:good::laugh:
worked for me too, set the chunk size to my system.img file in bytes (3221225472 in my case) to create 1 sparse file and then fastboot flashed it properly, finally got the device working after 1-2 hours of fiddling with it lol

 

[tal.aloni]

@fenixus:
Previous chunks are not erased when you flash the next chunk.

 

[fenixus]

@fenixus:
Previous chunks are not erased when you flash the next chunk.

well, it says "erasing system" before flashing each one ... anyways, that approach didn't work at all for me, phone was stuck on android logo after flashing the last chunk, factory reset, and reboot to system. maybe it differs between phones, mine is zenfone 2 laser ZE551KL.

 

[LeeArden]

hi tal.aloni,

i'm trying to flash a system.img (819MB, extracted via adb cat command) compressed with sparseconverter but it keeps failing. Here's what I've tried

1. Sparseconverter /compress system.img [path] 256MB
output files: 3 parts chunk1, chunk2, chunk3
result: "unknown chunk type"

2. Using the same parts from above:
I modified the headers of the parts from 3AFF26ED to ED26FF3A.
result: fastboot had no errors writing each part but the phone was still stuck, will not boot.

3. Sparseconverter /compress system.img [path] 819200000 (<---size of my phone's system partition)
output file: just 1 part
result: unknown chunk type

4. using the 1 part file from #3
modified the header from 3AFF26ED to ED26FF3A
result: "invalid sparse file format at header magi" but then fastboot continues with the erase, send and write
during the send part, fastboot twice reports "error: write_sparse_skip_chunk: don't care size 252907304 is not a multiple of the block size 4096", then it says OKAY and proceeds to writing which finally faild with "unknown chunk type"

Help please. Thank you.

 

[LeeArden]

hi tal.aloni,

i'm trying to flash a system.img (819MB, extracted via adb cat command) compressed with sparseconverter but it keeps failing. Here's what I've tried

1. Sparseconverter /compress system.img [path] 256MB
output files: 3 parts chunk1, chunk2, chunk3
result: "unknown chunk type"

2. Using the same parts from above:
I modified the headers of the parts from 3AFF26ED to ED26FF3A.
result: fastboot had no errors writing each part but the phone was still stuck, will not boot.

3. Sparseconverter /compress system.img [path] 819200000 (<---size of my phone's system partition)
output file: just 1 part
result: unknown chunk type

4. using the 1 part file from #3
modified the header from 3AFF26ED to ED26FF3A
result: "invalid sparse file format at header magi" but then fastboot continues with the erase, send and write
during the send part, fastboot twice reports "error: write_sparse_skip_chunk: don't care size 252907304 is not a multiple of the block size 4096", then it says OKAY and proceeds to writing which finally faild with "unknown chunk type"

Help please. Thank you.

Sorted it out, finally.

Might be useful info, Sparseconverter appends the correct header magic (3AFF26ED). The site I got it from got it wrong. But for the life of me I couldn't flash sparseconverter's output. Turned out I had to use another tool to pack the my raw system img to a sparse system img. Flashed the output with fastboot and fastboot automatically did the splitting. Phone works now.

 

[FullSignal]

I need help with .cs to .exe

HI, sorry but I'm not used to .cs fromat can you help me of how to make all files into one new .exe, because I already modified the sparseconvertehelper.cs file so now it say 512 instead of 4096, but I don't know how to run it.
greetings from Chile

Hi Matt,

The sector size for most flash devices is 4096 bytes.
The "compressed ext4 file system sparse image format" assumes that the image is going to be flashed to a flash device, and thus expects its size to be a multiple of 4096 bytes.
I'm not sure how you created this user data partition image, the size of a proper image (for flashing purposes) should be a multiple of 4096 bytes.
Edit: BlockSize can be set to 512 instead of 4096 in SparseConverter source code (SparseCompressionHelper.cs).

 

[tal.aloni]

@FullSignal
You should use Visual Studio (2005 or later) to compile the project into an exe.

 

[FullSignal]

no exe file

@FullSignal
You should use Visual Studio (2005 or later) to compile the project into an exe.

I am using it, in fact, I used it to edit the cs file, but it doesnt compile, there's no button, or option to do it, and in the microsoft support they say that in the folder named bin\debug I could find it, but there is nothing inside that folder...

 

[tal.aloni]

@FullSignal
You should open SpareConverter.sln with Visual Studio and then select Build > Build Solution from the menu, the exe will be saved to SparseConverter\bin\debug (or SparseConverter\bin\release if you've selected Release configuration).

 

[graj]

how to use it..

where is the setup of sparse converter i downloaded the zip but . while i run iit it automatically exits and close
provide a better solution for it

 

[tal.aloni]

@graj
It's a command line application, you need to run it from cmd.exe

 

[Azlun]

I just want to let you know again, even NOW on Android N 7.1 with Pixel XL, SparseConverter coded years ago is still working VERY well,
It makes me able to convert the dd raw data to sparse files,
Then, I can fastboot flash back my system image, it is much better than twrp-3.0.2-0-alpha2-fastboot-marlin backuped system file which always bring me boot loop...
1.backup code connected to TWRP adb：

adb shell "dd if=/dev/block/platform/soc/624000.ufshc/by-name/system_a of=/sdcard/img/system_a.img.raw bs=4096"
adb pull -p /sdcard/img/system_a.img.raw .\img\system_a.img.raw
sparseconverter /compress .\img\system_a.img.raw .\img 2147483648

2.fastboot restore image code with 1st raw fail raw_sparsechunk1
and success

D:\AzlunData2\ImageBackupCMD PixelXL EU>fastboot flash system img/system_a.img.raw
target reported max download size of 536870912 bytes
Invalid sparse file format at header magi
^C
D:\AzlunData2\ImageBackupCMD PixelXL EU>fastboot flash system img/system_a.img.raw_sparsechunk1
target reported max download size of 536870912 bytes
erasing 'system_a'...
OKAY [  0.419s]
sending sparse 'system_a' 1/3 (522477 KB)...
OKAY [ 12.813s]
writing 'system_a' 1/3...
OKAY [  8.930s]
sending sparse 'system_a' 2/3 (522467 KB)...
OKAY [ 12.908s]
writing 'system_a' 2/3...
OKAY [ 63.272s]
sending sparse 'system_a' 3/3 (474316 KB)...
OKAY [ 11.849s]
writing 'system_a' 3/3...
OKAY [ 62.526s]
finished. total time: 172.766s

Great progress! In my case
I generate my system.img.raw by
adb root
adb wait-for-device
adb shell "dd if=/dev/block/platform/f9824900.sdhci/by-name/system of=/sdcard/system.img.raw"
adb pull -p /sdcard/system.img.raw

1st Trail below will NOT works, but at the end you will see my solution:
Windows cmd log:
E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>sparseconverter /compress system.img.raw .\ 536870912
...
pause
E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_
sparsechunk1
target reported max download size of 536870912 bytes
erasing 'system'...
OKAY [ 0.591s]
sending 'system' (524284 KB)...
OKAY [ 1.350s]
writing 'system'...
OKAY [ 13.953s]
finished. total time: 15.897s

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_
sparsechunk2
target reported max download size of 536870912 bytes
erasing 'system'...
OKAY [ 0.532s]
sending 'system' (524287 KB)...
OKAY [ 1.351s]
writing 'system'...
OKAY [ 8.678s]
finished. total time: 10.565s

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_
sparsechunk3
target reported max download size of 536870912 bytes
erasing 'system'...
OKAY [ 0.524s]
sending 'system' (411515 KB)...
OKAY [ 1.075s]
writing 'system'...
OKAY [123.069s]
finished. total time: 124.671s

Please note even we pass the fastboot flashing, but it does NOT works after flash is finished, the phone can reboot to Linux logo, but block there, can not start Android Animation and go to desktop.
E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>adb shell
- exec '/system/bin/sh' failed: No such file or directory (2) -

SO, SOLUTION is:
as an conclusion, it seems fastboot can not manage the multi input images well, BUT it can handle a single compressed image files, so set the max image side to the partition size to 2684354560 can make fastboot works well.
I need to find my original dd output file system.img.raw in 2.5GB, then:
Windows cmd logs:

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>sparseconverter /compress system.img.raw .\ 2684354560

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_sp
arsechunk1
target reported max download size of 536870912 bytes
erasing 'system'...
OKAY [ 0.767s]
sending sparse 'system' (521371 KB)...
OKAY [ 4.530s]
writing 'system'...
OKAY [ 9.577s]
sending sparse 'system' (511116 KB)...
OKAY [ 6.465s]
writing 'system'...
OKAY [ 10.833s]
sending sparse 'system' (445892 KB)...
OKAY [ 6.274s]
writing 'system'...
OKAY [139.377s]
finished. total time: 177.831s

that will bring my system.img.raw back to my phone WELL
you can see my rooted su is there, ^_^:
E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>adb shell
[email protected]:/ $ su
su
[email protected]:/ #

 

[coolcoder001]

Sorted it out, finally.

Might be useful info, Sparseconverter appends the correct header magic (3AFF26ED). The site I got it from got it wrong. But for the life of me I couldn't flash sparseconverter's output. Turned out I had to use another tool to pack the my raw system img to a sparse system img. Flashed the output with fastboot and fastboot automatically did the splitting. Phone works now.

Hi ,

I am facing the similar issue. Could you please tell me how you resolved yours ?

How did you pack your raw system img to a sparse system img ?

My phone is bricked now.. and I would be really grateful to you if you could kindly show me the way to go .

 

[LeeArden]

coolcoder001,
hi, i did a ton of research and had this all memorized a couple of years ago, but i can't recall the details now. let me just search my old backups when i get home tonight.

 

[LeeArden]

The tool I referred to is ext2simg.exe, part of the Android IMG Repack Tools package here.

The resulting sparse img flashed successfully on my bricked tablet. Usage is via commandline and very straightforward.

I hope this helps unbrick your phone. Good luck!