[REF] ICS User Experience Graphs (sentiment analysis)

Which threads are the most useful to analyse?

  • ROM threads

    Votes: 40 87.0%

  • Kernel threads

    Votes: 24 52.2%

  • Theme/App threads

    Votes: 2 4.3%

  • General threads

    Votes: 4 8.7%

  • Threads for other devices

    Votes: 1 2.2%
  • Total voters
    46
Status
Not open for further replies.
Search This thread
By:
Advanced…
B

bedalus

Guest
UPDATE: [ROM | 4.0.4][MIUIV4] MIUIXPERIENCE #5 | Smooth! Duration! Stock!
[ROM][CM9] Brainmaster's CM9 & Black CM9 v3.0 OTA
[ROM][MIUI 4][2.4.20] Catalyst MIUI with OTA
[ROM] Next ICS |STOCK|ELEGANT|INVERTED| CM9 Kernel

all added to the UX site.

I found some of the threads too short to get a reliable recent UX, so I skipped them (e.g. Pool Party).

I will have a look through the development section to see if any new 4.0.4 ROMs have arrived, and if they have had sufficient enough posts to get a usable UX benchmark.
 
tchaari

tchaari

Senior Member
Dec 27, 2011
1,124
1,104
bedalus said:
I think crossbones got a good score with mine because it is a very short thread. Usually a new thread begins with a lot of well-wishers posting encouragement. It's strange how you got a low score for it. Anyway, I compared some of our mutual scores:

--- ROM Title = Tchaari's method / my method

CyanogenMod 9 v4.0.4 v7.0 + CM Night builds=0% / 0%

Android Open Kang Project-build 22=+19.4% / +44.5%

Slim ICS 68MB- Clean, - The Google Way=+54.8% / +67.1%

I compared your most recent to my most recent.

Assuming that wingify has 50% as neutral, I've calculated the proportional improvement of AOKP and Slim ICS compared to Cyanogenmod using your method and my method. (There were only three ROMs that we both did).

I looks like our methods produce comparable results.

EDIT: I'm see if I can get a key for wingify and see if I can use curl for their api like I do with alchemyapi.

EDIT2: yeah... curl works fine. I may need a key from them first though.

EDIT3: Okay, I tried a brief test with wingify (I had previously tried it before with AlchemyAPI)

With AlchemyAPI
"really good" - 60.3% (positive)
"really not good" - 42% (negative) (50% is neutral)

With Wingify
"really good" - 91% (highly positive)
"really not good" - 91% (highly positive)

So it looks like Wingify may have some issues with negation.
Click to expand...
Click to collapse

Nice analysis bedalus ;)
It seems that AlchemyAPI is better for small strings but not really for a big amount of text...

This is what I obtained by using Alchemy in the place of Wingify. The latter seems to be a bit closer to reality. Do you share the same opinion bedalus?


1) [ROM][IMM76D][19/04/12] Next ICS |STOCK|ELEGANT|INVERTED| CM9 Kernel | [20th February 2012 03:55 PM : 0,166] [20th February 2012 03:55 PM : 0,166] [30th March 2012 10:32 AM : 0,209] [7th April 2012 05:30 AM : 0,086] [26th April 2012 11:20 AM : 0,067] --> {avg=0,139}
2) [ACS][ROM][4.0.4 NS 3G][GSM] Pool Party V .5 ICS [20th April 2012 08:05 PM : 0,127] [20th April 2012 08:05 PM : 0,127] --> {avg=0,127}
3) [ROM-CRESPO] [AOSP] [ICS] Crossbones v0.3.1 [04/21] [18th January 2012 05:36 AM : 0,101] [18th January 2012 05:36 AM : 0,101] [29th January 2012 03:22 PM : 0,096] [8th April 2012 05:58 PM : 0,1] --> {avg=0,1}
4) [ROM][DEV][PRE-ALPHA] HTC Sense 4.0 | ICS 4.0.3 | 2/17/12 [19th February 2012 10:59 PM : 0,092] [7th March 2012 10:48 PM : 0,143] [17th March 2012 10:46 AM : 0,108] [1st April 2012 09:05 AM : 0,061] [23rd April 2012 03:27 PM : 0,094] --> {avg=0,1}
5) [ROM][PORT] Nexus Sense 3G v1.4 | Full HTC Sense Experience | 03/04/12 [6th March 2012 06:35 PM : 0,093] [18th March 2012 02:39 PM : 0,122] [4th April 2012 09:19 AM : 0,014] [14th April 2012 07:32 PM : 0,099] [24th April 2012 01:38 PM : 0,104] --> {avg=0,086}
6) [ROM][MIUI 4][2.4.20] Catalyst MIUI with OTA [13th April 2012 03:08 PM : 0,001] [20th April 2012 04:59 PM : 0,142] [24th April 2012 03:30 PM : 0,115] [25th April 2012 04:56 PM : 0,017] [26th April 2012 11:22 AM : 0,095] --> {avg=0,074}
7) [ROM][4.0.4]Slim ICS 68MB- Clean, Simple & Fast - The Google Way [1st April 2012 09:59 PM : 0,032] [5th April 2012 12:28 AM : 0,061] [14th April 2012 10:21 AM : 0,079] [23rd April 2012 09:04 AM : 0,088] [25th April 2012 01:26 PM : 0,075] --> {avg=0,067}
8) [ROM] Ultimate ICS v4.0 [1st December 2011 01:21 PM : 0,045] [22nd December 2011 12:04 PM : 0,061] [26th December 2011 06:45 PM : 0,102] [12th January 2012 07:53 PM : 0,077] [25th April 2012 07:35 PM : 0,049] --> {avg=0,067}
9) [ROM] Androxide 2.0.0 | AOSP | 4.0.4 | 24-04-2012 [26th March 2012 09:52 AM : 0,035] [29th March 2012 03:52 PM : 0,13] [10th April 2012 06:41 AM : 0,034] [25th April 2012 05:26 PM : 0,078] [26th April 2012 11:35 AM : 0,052] --> {avg=0,066}
10) [ROM][ICS][AOSP] Bugless Beast (Nightlies) [11th April 2012 01:44 PM : 0,095] [13th April 2012 05:18 PM : 0,084] [16th April 2012 11:52 PM : 0,083] [23rd April 2012 07:57 PM : 0,056] [26th April 2012 11:37 AM : 0,005] --> {avg=0,064}
11) [ROM] [i9020/23] CyanogenMod 9 v4.0.4 v7.0 + CM Night builds [23rd April 2012 10:43 AM : 0,03] [23rd April 2012 08:09 PM : 0,048] [24th April 2012 02:33 PM : 0,078] [25th April 2012 09:49 PM : 0,064] --> {avg=0,055}
12) [ROM][AOSP][4.0.4][Crespo] Gummy 102 - 21 APR 2012 [14th April 2012 10:24 AM : 0,09] [17th April 2012 07:46 PM : 0,048] [21st April 2012 04:17 PM : 0,058] [25th April 2012 08:31 AM : 0,022] --> {avg=0,055}
13) [ROM][INFO][AOSP][ICS] Android Open Kang Project-build 22 [12th March 2012 03:13 PM : 0,06] [28th March 2012 07:54 PM : -0,012] [16th April 2012 04:32 AM : 0,1] [24th April 2012 12:44 AM : 0,101] [25th April 2012 11:29 AM : 0,022] --> {avg=0,055}

---------- Post added at 11:52 AM ---------- Previous post was at 11:50 AM ----------

@bedalus, if you find other free sentiment analysis APIs, please let me know to test them and compare them to what we are using now ;)
 
B

bedalus

Guest
tchaari said:
Nice analysis bedalus ;)
It seems that AlchemyAPI is better for small strings but not really for a big amount of text...
Click to expand...
Click to collapse
I think if it works better on a small string, it works better on all text!

tchaari said:
1) [ROM][IMM76D][19/04/12] Next ICS |STOCK|ELEGANT|INVERTED| CM9
3) [ROM-CRESPO] [AOSP] [ICS] Crossbones v0.3.1
5) [ROM][PORT] Nexus Sense 3G v1.4 | Full HTC Sense Experience |
6) [ROM][MIUI 4][2.4.20] Catalyst MIUI with OTA
9) [ROM] Androxide 2.0.0 | AOSP | 4.0.4 | 24-04-2012
10) [ROM][ICS][AOSP] Bugless Beast (Nightlies)
11) [ROM] [i9020/23] CyanogenMod 9 v4.0.4 v7.0 + CM Night builds
13) [ROM][INFO][AOSP][ICS] Android Open Kang Project-build 22
Click to expand...
Click to collapse
I will include 3G Sense and Androxide in the next update (also gummy). But from now on, only threads being actively developed and with at least 400 replies will be included. In fact I may take crossbones out because it has very few posts, and I think the developer has had a new baby (=0 time for dev work!)

My mutual ranking is
Next ICS
Catalyst MIUI
AOKP
BB
Cyanogenmod

So only AOKP out of place (higher up in mine... but I changed the way I rank now, a simple addition of entire thread UX + recent UX)

tchaari said:
@bedalus, if you find other free sentiment analysis APIs, please let me know to test them and compare them to what we are using now ;)
Click to expand...
Click to collapse

I found this list: https://docs.google.com/spreadsheet/ccc?key=0Ajqw0BPB53J7dFZmTGhCeGFvNzBGazliQXRlYnBfeFE

Some of them might have API/Curl for free. I'm happy with AlchemyAPI for now.
 
tchaari

tchaari

Senior Member
Dec 27, 2011
1,124
1,104
Cool, let's keep the two rankings then. Two different approaches :)
I'll, try to generate the graph script when i have some free time. Can you please pm me your graph script bedalus?

Sent from my Nexus S using XDA
 
B

bedalus

Guest
tchaari said:
Cool, let's keep the two rankings then. Two different approaches :)
I'll, try to generate the graph script when i have some free time. Can you please pm me your graph script bedalus?
Click to expand...
Click to collapse

I'll just attach a spreadsheet here. It's easier. Is this what you need?
 

Attachments

  • chart_formulas.xlsx
    10 KB · Views: 5
tchaari

tchaari

Senior Member
Dec 27, 2011
1,124
1,104
bedalus said:
I'll just attach a spreadsheet here. It's easier. Is this what you need?
Click to expand...
Click to collapse

Thanks buddy. Can you please give me an idea about how did you integrate the graph in your webpage?
EDIT: Btw, I am loving your webpage bedalus. Really nice work ;)
 
Last edited:
B

bedalus

Guest
tchaari said:
Thanks buddy. Can you please give me an idea about how did you integrate the graph in your webpage?
Click to expand...
Click to collapse

It's my first ever web page!

I made one with Google sites. All you have to do is import the chart Not sure, but probably it has to be a chart in a Google doc, so email that excel doc to yourself first, then open it as a Google doc. Paste in your own data, then create a chart. You'll be able to import it from your Google site.

kernels ; battery ; ROM ; gov/sched
Now with summaries in the first posts. Convenient for XDA app users!
 
B

bedalus

Guest
Some kernels now up: http://goo.gl/i466T

Trinity will have to wait until tomorrow, and there is also GlaDOS (just ran out of API allowance part-way into the trinity thread).
 
stewarta

stewarta

Senior Member
Jun 8, 2004
422
37
Melbourne
bedalus said:
PLEASE VOTE ABOVE! (Multichoice)
View the ROMS: Here.

View the Kernels: Here.
You can hover over the bars to see which thread it is, or if you are using a phone, just tap on them.

>>> F.A.Q. <<<
[Q] What does this tell me?
[A] If you are looking for a new ROM, you might want to start with one that gets positive feedback.
[Q] How do you rank 'a feeling'?
[A] A sophisticated API is used to perform sentiment analysis.
[Q] Sounds a bit technical...
[A] See post #2 for more of the technical details, links, etc.
[Q] Can you rank MegaFace's SuperNutty ROM v256.0 thread please?
[A] Sure. But it needs to have at least 400 replies.

Where did the other benchmarks go?

All ICS ROM Benchmarks: this thread

Battery Drain Benchmarks: this thread

Kernel Features and Benchmarks: this thread

CPU Governors and I/O Schedulers: this thread

Power Saving Governors: this thread

Does SuperCharging work?: this thread
Click to expand...
Click to collapse

Good stuff. I'd still like to put the posts through Leximancer and see the output. I can explain via PM if you are interested. ;)
 
B

bedalus

Guest
UPDATE: Trinity and GlaDOS added.

Trinity ate three quarters of my API allowance today, GlaDOS the other quarter. But I don't think there is much else I can add at the moment anyway.

I removed the average and recent bar charts for the ROMs, leaving only the combined bar chart to make things simpler (and I kept the line-graph because it looks cool...)

There are now links to the data in its original numerical form (on google docs spreadsheets) at the bottom of the sidebar.

I plan to update all the ROMs and Kernels weekly.

EDIT: I see I have recognised contributor status! Thanks community! :D
 
Last edited:
B

bedalus

Guest
UPDATE: Apex added to the NSUX chart. (bigxie has pushed out a 4.0.4 test version)

Also, put all the ROMs in rank order.
 
B

bedalus

Guest
UPDATE: Performed my first attempt at a global update (all ROM and kernel threads)

...discovered a bug in my program, corrected it (thank goodness for having the sense to backup all of my working folder!)

...after that, the update went smoothly!

Currently the bars show the global average plus the most recent 10%. I think I will change this to add half the most recent score, a quarter of the one before that, an eighth of the one before that... etc. I think this is a better way of ensuring the results reflect current sentiment.
 
tchaari

tchaari

Senior Member
Dec 27, 2011
1,124
1,104
bedalus said:
UPDATE: Performed my first attempt at a global update (all ROM and kernel threads)

...discovered a bug in my program, corrected it (thank goodness for having the sense to backup all of my working folder!)

...after that, the update went smoothly!

Currently the bars show the global average plus the most recent 10%. I think I will change this to add half the most recent score, a quarter of the one before that, an eighth of the one before that... etc. I think this is a better way of ensuring the results reflect current sentiment.
Click to expand...
Click to collapse

bedalus, there is a problem at the ROM graph display. The graph isn't shown and an error message is displayed (All series on a given axis must be of the same data type).
 
B

bedalus

Guest
Oops technical difficulties...

...and we're back!
 
Last edited:
Status
Not open for further replies.

Similar threads

oldblue910
[REF] ALL Nexus S OTA Updates for ALL Variants, Plus Full ROM Packages
Replies
519
Views
389K
EthanFirst
EthanFirst
Vandam500
[REF] Complete List of Roms | Kernels | Rooting/Other Guides for Nexus S (GSM)
Replies
164
Views
256K
KBanause
KBanause
B
  • Locked
  • Poll
[REF] Kernel Features and Benchmarks
Replies
456
Views
102K
af974
af974
oldblue910
[I9020T][I9023][I9020A][M200][D720] Nexus S 4.1.1/JRO03E Jelly Bean Update Thread
Replies
3K
Views
979K
punjabijack
punjabijack
B
  • Locked
  • Poll
[REF] All ICS ROMs Benchmarked
Replies
877
Views
143K
Andy
Andy

Top Liked Posts

24 Hours All time
  • There are no posts matching your filters.
  • 3
    Oodie
    Oodie
    tchaari said:
    enjoy your week buddy and don't think a lot about benchmarks in your holidays :D
    Click to expand...
    Click to collapse

    He gets sweet dreams & also nightmares of " BENCHMARKS " .

    As i heard from one of bedalus neighbor , Sometimes when bedalus sleep he says ,

    1. " Oh ! my dear benchmark . my love , ***** ( adult only words occurs after that )

    2. " Ohh ! what the hell benchmark , you go to hell , you f**** ( adult only words occurs after that )

    Then he wakes up & runs to his PC & so on , so on
    icon10.gif


    Edit : Enjoy your holiday buddy ! :D
    View
    2
    glennkaonang
    glennkaonang
    I think bedalus & tchaari should get the title "Recognized Analyst" here :D :D

    Sent from my Nexus S using xda premium
    View
    2
    tchaari
    tchaari
    This is the new ranking of the ROMs according to my sentiment analysis program. I removed the ROMs that were not updated since one month and I also removed the threads that have less than 200 posts.
    1) [ROM] Androxide 2.0.0 | AOSP | 4.0.4 | 24-04-2012 [26th March 2012 09:52 AM : 0,866] [29th March 2012 03:52 PM : 0,881] [10th April 2012 06:41 AM : 0,637] [25th April 2012 05:26 PM : 0,783] [30th April 2012 11:00 PM : 1,008] --> {avg=0,835}
    2) [ROM | 4.0.4][MIUIV4] MIUIXPERIENCE #5 | Smooth! Duration! Stock! [20th April 2012 05:57 PM : 0,821] [23rd April 2012 12:44 PM : 0,803] [25th April 2012 04:57 AM : 0,822] [28th April 2012 07:39 PM : 0,765] --> {avg=0,803}
    3) [ROM][ICS][AOSP] Bugless Beast (Nightlies) [13th April 2012 05:18 PM : 0,748] [16th April 2012 11:52 PM : 0,857] [23rd April 2012 07:57 PM : 0,773] [1st May 2012 12:33 AM : 0,785] --> {avg=0,791}
    4) [ROM][4.0.4]Slim ICS 68MB- Clean, Simple & Fast - The Google Way [5th April 2012 12:28 AM : 0,846] [14th April 2012 10:21 AM : 0,857] [23rd April 2012 09:04 AM : 0,681] [30th April 2012 03:43 PM : 0,652] --> {avg=0,759}
    5) [ROM][MIUI 4][2.4.27] Catalyst MIUI with Language Packs and OTA Update [20th April 2012 04:59 PM : 0,502] [24th April 2012 03:30 PM : 0,603] [25th April 2012 04:56 PM : 0,763] [29th April 2012 12:02 PM : 0,758] [1st May 2012 01:09 AM : 0,878] --> {avg=0,701}
    6) [ROM][INFO][AOSP][ICS] Android Open Kang Project-build 22 [28th March 2012 07:54 PM : 0,794] [16th April 2012 04:32 AM : 0,795] [24th April 2012 12:44 AM : 0,855] [29th April 2012 12:34 PM : 0,807] [30th April 2012 01:20 PM : 0,185] --> {avg=0,687}
    7) [ROM][AOSP][4.0.4][Crespo] Gummy 102 - 21 APR 2012 [14th April 2012 10:24 AM : 0,668] [17th April 2012 07:46 PM : 0,605] [21st April 2012 04:17 PM : 0,741] [25th April 2012 08:31 AM : 0,673] [29th April 2012 03:40 PM : 0,687] --> {avg=0,675}
    8) [ROM] [i9020/23] CyanogenMod 9 v4.0.4 v7.0 + CM Night builds [24th April 2012 02:33 PM : 0,7] [25th April 2012 09:49 PM : 0,763] [27th April 2012 01:55 PM : 0,536] [29th April 2012 10:14 PM : 0,721] [30th April 2012 10:47 PM : 0,644] --> {avg=0,673}
    View
    2
    tchaari
    tchaari
    Oodie said:
    with my Data limit for a month ( only 3Gb ) i really can't release any roms frequently . That's the reason :( I'm thinking of quiting from rom modding & keep on testing other roms & kernels :rolleyes: :D
    Click to expand...
    Click to collapse

    Oh no, please don't. Don't worry, I can wait an entire month for the next Oodie Skank :D
    Really Oodie, if you have enough free time for developing and you really like it, I highly encourage you to keep modding. I would like to do it but not enough free time (teaching, scientific research, student projects supervising, administrative tasks, wife, family...) and I really feel happy when young devs like you do things that I wanted to do ;)
    View
    1
    tchaari
    tchaari
    glennkaonang said:
    I think bedalus & tchaari should get the title "Recognized Analyst" here :D :D

    Sent from my Nexus S using xda premium
    Click to expand...
    Click to collapse

    You are also in the same boat Glenn. The ICS ROM hierarchy is the proof ;)
    Bedalus is doing the hard work here. I just try to help by some ideas. I hope I had more free time to implement these ideas (that aren't always interesting lol) ;)
    View

New posts