[Android 7.0+] AppRadio Unchained Rootless

Search This thread

Area51©

Senior Member
Aug 30, 2013
2,482
1,061
Breda
Looking great. Test some other app besides YouTube next time, please. Great work.

Edit
Does the audio sound from the phone also? It fells like both are sounding. Lol. Scrolling is a bit laggy but I think it is awesome what you've accomplished.
I used VLC and YouTube as this will show the video performance the best way. If those are not laggy, any app should work. What apps would you like to see?

Audio of the phone was on but very soft. I don't think you can hear it. Anyway, audio has some lag, it's about 500 milliseconds. I did some analysis on it and it seems to be the AudioRecord class from the Android API itself. That's on my old Nexus 6. Still have to see how it's on a more recent phone. I tested this also with Android Auto itself and Screen2Auto on a Nexus 5X , it gives about 300 milliseconds delay.

Touch is a bit laggy because I am using an accessibility service to inject touch. This allows only to inject complete gestures so the app is recording a gesture from the head unit and then playing it back on the phone. This is the only way to inject touch without root. I wanted to have this working first as most users won't have a rooted phone.
When root is available, touch events can be injected directly and there won't be hardly any delay. The plan is to include this also in the app.
 

tiho5

Senior Member
Jun 21, 2010
2,639
999
Bulgaria
I used VLC and YouTube as this will show the video performance the best way. If those are not laggy, any app should work. What apps would you like to see?

Audio of the phone was on but very soft. I don't think you can hear it. Anyway, audio has some lag, it's about 500 milliseconds. I did some analysis on it and it seems to be the AudioRecord class from the Android API itself. That's on my old Nexus 6. Still have to see how it's on a more recent phone. I tested this also with Android Auto itself and Screen2Auto on a Nexus 5X , it gives about 300 milliseconds delay.

Touch is a bit laggy because I am using an accessibility service to inject touch. This allows only to inject complete gestures so the app is recording a gesture from the head unit and then playing it back on the phone. This is the only way to inject touch without root. I wanted to have this working first as most users won't have a rooted phone.
When root is available, touch events can be injected directly and there won't be hardly any delay. The plan is to include this also in the app.
Great.

Can you make audio only for the head unit?
May be also try a simple other app. Like say phone or messages. Just for a quick example.

You are doing a fantastic product. Thank you!
 

arimhan

New member
May 31, 2022
3
0
This looks already daily drivable.
One thing that could be good would be transmisting navigation data to the car dashboard. On my car when using android auto, the car also display the next turn on the dashboard too but that may be hard to do if you don't have a dual screen android auto head unit.
 

Area51©

Senior Member
Aug 30, 2013
2,482
1,061
Breda
This looks already daily drivable.
One thing that could be good would be transmisting navigation data to the car dashboard. On my car when using android auto, the car also display the next turn on the dashboard too but that may be hard to do if you don't have a dual screen android auto head unit.
It's not daily driveable yet by a long shot. It only has enough implemented to work with this head unit. It can only connect once to the head unit. When the connection fails, the app has to be killed first and then restarted. There is also different protocol versions. This only supports one version partly. So there's quite some work to do.

This version was meant only to evaluate if it's possible to get video, audio and touch working without any major obstacles. Kind of a go or no go for further development. It seems there's no major snags so I will continue.

One of the things I will try to add is a Network to USB SSL decryption proxy that can work as a man in the middle between Android Auto and the head unit. This will allow to decrypt and log all the communication between AA and the HU.

Regarding sending navigation data to the dashboard, It would likely be possible to discover how to send that data to the HU. However that navigation app will only do that when it is running in Android Auto mode. When it's running mirrored via AAUnchained, it will assume it's running on a phone screen and not send that data.
 

bigcid10

Senior Member
Dec 17, 2011
515
73
West Palm Beach
For the head unit it will look like Android Auto is connected. On the phone side nothing from the built-in Android Auto stuff is used. When connecting via USB, the phone will ask to use which app you want to use AA or AAUnchained and you can set to always use the app you select. As for wireless, I don't know yet, still have to find out how the official connection method does work.
Ok, that's fine
Thank you
 

maximushugus

Member
Apr 15, 2017
32
4
I'm following your work with a lot of attention. I looks very very promising.
I have a question : will the app work without Android Auto installed on the phone ? Will it work without Google Services ?
Android Auto is the only Google app that is keeping me from running a complete deGoogle Lineage OS phone, so it would be awsome !
Thanks
 

Area51©

Senior Member
Aug 30, 2013
2,482
1,061
Breda
I'm following your work with a lot of attention. I looks very very promising.
I have a question : will the app work without Android Auto installed on the phone ? Will it work without Google Services ?
Android Auto is the only Google app that is keeping me from running a complete deGoogle Lineage OS phone, so it would be awsome !
Thanks
Yes, the app implements the Android Auto protocol internally so it works without Google Play Services.
 

Area51©

Senior Member
Aug 30, 2013
2,482
1,061
Breda
I continued working on AAUnchained for another month.
Short progress report:

General:
- App can now completely reset to initial state so it can be started again
- Implemented start / stop when USB to head unit connected / disconnected
- Powerlock added so phone will never sleep when AAUnchained is active

Protocol
- Did some more testing with different emulators regarding different protocol versions
- Made lots of sensor status messages (like day/night, odometer etc.) working
- Video and audio focus messages are now supported which means that you can now jump back to the head unit and return to the mirrored phone again without losing the connection

Touch improvements:
- Multitouch is now working
- Touch compensates for black bars because of different aspect ratio between head unit and phone

More testing:
- Bought a Samsung S22+ for testing with Android 12 & high performance hardware
- No artefacts in video anymore (there were some on my old Nexus 6)
- Audio delay is down to 200 milliseconds (was 800 - 1000 on Nexus 6)

I am going to make a new demo video soon.
 

stargate125645

Senior Member
Sep 25, 2012
774
70
Omaha, NE
I continued working on AAUnchained for another month.
Short progress report:

General:
- App can now completely reset to initial state so it can be started again
- Implemented start / stop when USB to head unit connected / disconnected
- Powerlock added so phone will never sleep when AAUnchained is active

Protocol
- Did some more testing with different emulators regarding different protocol versions
- Made lots of sensor status messages (like day/night, odometer etc.) working
- Video and audio focus messages are now supported which means that you can now jump back to the head unit and return to the mirrored phone again without losing the connection

Touch improvements:
- Multitouch is now working
- Touch compensates for black bars because of different aspect ratio between head unit and phone

More testing:
- Bought a Samsung S22+ for testing with Android 12 & high performance hardware
- No artefacts in video anymore (there were some on my old Nexus 6)
- Audio delay is down to 200 milliseconds (was 800 - 1000 on Nexus 6)

I am going to make a new demo video soon.
Is the aspect ratio going to be an issue going forward or did you figure that out, too? It's not something I even thought about. You definitely have me giddy for the first beta!
 
Last edited:

Area51©

Senior Member
Aug 30, 2013
2,482
1,061
Breda
Is the aspect ratio going to be an issue going forward or did you figure that out, too? It's not something I even thought about. You definitely have me giddy for the first beta!
The AA supported resolutions have 5 : 3 aspect ratio. Most modern phones have aspect ratio like 18.5 : 9 or 19.5 : 9 which means that there will be black bars on top/ bottom when screen mirroring. I just made it so that those black bar areas are excluded from touch and only the visible area is used.
 

Area51©

Senior Member
Aug 30, 2013
2,482
1,061
Breda
Recently I encountered another issue that is really annoying. It's with audio capturing. While Google now (in Android 10 or higher) allows audio capturing it's limited to certain audio types. Those are USAGE_MEDIA, USAGE_GAME and USAGE_UNKNOWN.

Apps like Youtube and VLC use USAGE_MEDIA for audio so that can be captured. However navigation apps like Google maps or TomTom Go use another audio type for voice instructions and that audio can't be captured.

It's like a dealbreaker to me. I think the most important thing that you want to use is a navigation app so that should work as intended. Probably voice instructions can play from the phone speaker but that would be just lame.

I have been looking a way to capture all audio but it can be only done by system apps. Currently that means that app has to be signed with the platform keys of the installed Android version. This means that even on a rooted phone it can't be done. Doesn't look like there's a solution.
 

stargate125645

Senior Member
Sep 25, 2012
774
70
Omaha, NE
The AA supported resolutions have 5 : 3 aspect ratio. Most modern phones have aspect ratio like 18.5 : 9 or 19.5 : 9 which means that there will be black bars on top/ bottom when screen mirroring. I just made it so that those black bar areas are excluded from touch and only the visible area is used.
For my edification, why wouldn't there be a way to make the interface take up the entire screen like AA does currently? Or like what Samsung phones do to resolution when connecting via MiraCast ("match source" I think the option is called)?

As for your audio capture post, that is definitely a bummer. How does AA do that now?
 

Area51©

Senior Member
Aug 30, 2013
2,482
1,061
Breda
For my edification, why wouldn't there be a way to make the interface take up the entire screen like AA does currently? Or like what Samsung phones do to resolution when connecting via MiraCast ("match source" I think the option is called)?
Samsung just sets another screen resolution. A normal app can't do that. With root access it can be done but it will have side effects. For instance I do that on my Android TV box and then the default launcher app can't handle it. Samsung can integrate everything nicely but I can't do that.

Another option may be to create a virtual screen. But I don't know if you can make any app to display on that. Even if so it would require to make some kind of app launcher and it's a lot of work.

As for your audio capture post, that is definitely a bummer. How does AA do that now?

It's a matter of app permissions. And Google being Google just made sure that their apps can have every permission that exists. Only the core part of Android is open source. More and more stuff is closed source from Google.
Google dictates that a normal app can't capture system audio.
 

stargate125645

Senior Member
Sep 25, 2012
774
70
Omaha, NE
Samsung just sets another screen resolution. A normal app can't do that. With root access it can be done but it will have side effects. For instance I do that on my Android TV box and then the default launcher app can't handle it. Samsung can integrate everything nicely but I can't do that.

Another option may be to create a virtual screen. But I don't know if you can make any app to display on that. Even if so it would require to make some kind of app launcher and it's a lot of work.



It's a matter of app permissions. And Google being Google just made sure that their apps can have every permission that exists. Only the core part of Android is open source. More and more stuff is closed source from Google.
Google dictates that a normal app can't capture system audio.
I'm sorry! This seems like a big obstacle. 😩
 

Area51©

Senior Member
Aug 30, 2013
2,482
1,061
Breda
Ok, I did some more investigation on this. There is a way to capture system audio on a rooted phone. I didn't verify this on Android 12 yet but on Android 11 it's possible. Also it's a somewhat complicated workaround.
The huge disadvantage is that of course the requirement for the phone to be rooted, limits to number of potential customers enormously.
My initial requirement was that everything should work on any phone, not just rooted ones. So now I am pondering whether to continue or not. There's still a lot of work to do to finish it. Don't want all this work to be wasted.
 

stargate125645

Senior Member
Sep 25, 2012
774
70
Omaha, NE
Ok, I did some more investigation on this. There is a way to capture system audio on a rooted phone. I didn't verify this on Android 12 yet but on Android 11 it's possible. Also it's a somewhat complicated workaround.
The huge disadvantage is that of course the requirement for the phone to be rooted, limits to number of potential customers enormously.
My initial requirement was that everything should work on any phone, not just rooted ones. So now I am pondering whether to continue or not. There's still a lot of work to do to finish it. Don't want all this work to be wasted.
I stopped caring about rooting long ago as enough features were added to Android to avoid the need, coupled with your ARUR app. Would it be possible for you to make this work with an Android stick and not a phone - presumably something that's easier to root?

Also, for my edification, what allows ARUR to capture all audio but not your AA facsimile?
 

Area51©

Senior Member
Aug 30, 2013
2,482
1,061
Breda
I stopped caring about rooting long ago as enough features were added to Android to avoid the need, coupled with your ARUR app. Would it be possible for you to make this work with an Android stick and not a phone - presumably something that's easier to root?
Yes but Android sticks don't really exist anymore. Mostly it's TV boxes and those have several issues that make them less easy to use. I know because I am using one with my old AppRadio in one of my cars. Besides there's already devices from China that are kind of Android sticks that have an Android Auto interface. Kind of expensive for the hardware that they provide though.

Also, for my edification, what allows ARUR to capture all audio but not your AA facsimile?
ARUR doesn't have to capture anything. That's because there's extra hardware involved like a USB-C to HDMI adapter or a Miracast adapter. When supported, the manufacturer makes sure that audio and video is properly routed to that adapter.
In case of AAUnchained it's pure software so the app needs to be able to capture video and audio itself to be able to stream it via USB or Wifi. Unfortunately user installed apps have less permissions than system apps. That's the reason not all audio can be captured.

One possible solution is for instance to create a simple device that can act as a Bluetooth audio receiver which then can then transfer the audio back to the phone via Wifi. Possibly in collaboration with AAWireless because this already has the required hardware and can do it in software as a side job. But I prefer a software-only solution.
 

tiho5

Senior Member
Jun 21, 2010
2,639
999
Bulgaria
Ok, I did some more investigation on this. There is a way to capture system audio on a rooted phone. I didn't verify this on Android 12 yet but on Android 11 it's possible. Also it's a somewhat complicated workaround.
The huge disadvantage is that of course the requirement for the phone to be rooted, limits to number of potential customers enormously.
My initial requirement was that everything should work on any phone, not just rooted ones. So now I am pondering whether to continue or not. There's still a lot of work to do to finish it. Don't want all this work to be wasted.
Don't quick so far ahead. I will never use a non rooted phone and many are like that.
 

Top Liked Posts

  • There are no posts matching your filters.
  • 7
    Great news regarding AAUnchained. Finally video streaming to my Pioneer Android Auto head unit is working and it is really stable. See here for a demo: AAUnchained video mirroring demo.
    This demo shows the AAUnchained app running on my old Nexus 6 with Android 9. It's connected wirelessly to the head unit via an AAWireless adapter. AAunchained mirrors the phone's screen to the head unit directly, 'talking' the Android Auto protocol. Because Android Auto is not used, all of the screen estate is available for mirroring.
    Next things to do is to get audio working and implementing touch from the head unit to the phone.

    I did some more testing with audio and it's also sent to the head unit using the AA protocol. Capturing internal audio is only possible with Android 10 and higher so that's going to be a requirement for the app. Have to see how that works but it's not nearly as difficult as getting the video to work.

    Touch is even simpler. Touch info is already sent via the AA protocol, just a matter of hooking it up.
    6
    Now have audio working as well. Going to add touch and then make a full demo.
    4
    AppRadio Unchained Rootless allows full mirroring of your phone from your AppRadio. This means that any app can be controlled from the head unit screen and not just a few that are specially adapted.

    AppRadio Unchained Rootless is available from the Play store here: AppRadio Unchained Rootless
    The user's manual is available here: User's manual

    For this app to work Android 7 or higher is required. As Android 7 only allows to inject complete gestures, a gesture has to be completed first on the head unit before it will be dispatched to the phone. It works similar to record and playback. Suppose you need to do a long press of 2 seconds, first press for 2 seconds, once you lift your finger it will be dispatched and replicated on the phone where it will take 2 seconds as well. It's recommended to only do things that take a short time so there won't be too much delay.

    Important
    The 'Smartphone setup' on the head unit needs to be set correctly for Android as by default it's configured for Iphone. Go to Settings->System->Input/Output Settings->SmartphoneSetup and set Device to 'Others' and Connection to 'HDMI'. See this video: Smartphone setup

    Any other AppRadio related app needs to be uninstalled as this blocks the connection to AppRadio Unchained Rootless.

    AppRadio mode requires that your device is connected to the HDMI input of the head unit. Depending on the device this can be done with a MHL / Slimport / Miracast / Chromecast adapter. This app supports automatic connection to wireless screencasting devices. Since Google API doesn't support this directly it is done via the phone's GUI. Note that only the phone's built-in screencasting capabilities can be used.

    Because this app may not work for your setup there's an extended trial period of 48 hours. In order to claim this simply request a refund within 48 hours after purchase by emailing the order number to the support email address.

    Supported head units: any AppRadio that supports Android AppMode via HDMI.
    For example: SPH-DA100, SPH-DA110, SPH-DA210, SPH-DA120, AVH-X8500BHS, AVH-4000NEX, AVH-4100NEX, AVH-4200NEX, AVIC-X850BT, AVIC-X950BH, AVIC-Z150BH, AVIC-6000NEX, AVIC-6100NEX, AVIC-6200NEX, AVIC-7000NEX, AVIC-7100NEX, AVIC-7200NEX, AVIC-8000NEX, AVIC-8100NEX, AVIC-8200NEX

    Units that have AppRadio mode via USB (a.k.a. AppRadio One) are not supported.

    Android 7 Bluetooth bug
    If during connection 'Accept thread error' is displayed this is not because of a bug in the app but due to a bug in Android 7.
    It can be fixed by disabling BT background scanning: Go to Settings -> Location, in top right menu click Scanning -> Bluetooth Scanning.

    The following features are supported:
    - Multitouch
    - AppRadio buttons
    - GPS data transfer via mock locations (only works with head units that have a GPS receiver)
    - Wake lock
    - Rotation locker (to put any app in landscape mode)
    - Real calibration
    - Start on HDMI detection (for use with phones and HDMI adapters)
    - Notifications to indicate connection state
    - Diagnostics
    - Automatic Bluetooth toggle for improved connection

    Beta test version
    The beta test version has the latest features but can also contain bugs.
    You need to become a tester of the AppRadio Unchained Rootless app to be able to get it.
    Please apply here: https://play.google.com/apps/testing/mars.area51.arunchainednr

    Beta test release notes
    0.09
    - Fix for black menu text color, now white as it should be.
    - SWC commands can now be directed to a target app.

    0.08
    - Fix for calibration out of range.
    - Fix for delay of casting start.
    - Support for automatic casting on Samsung Android 9.
    - Fix for startForegroundService.
    - Fix for rotation locker.
    - Change HDMI wired connection trigger to be triggered by Bluetooth connection.

    AppRadio is a registered trademark of Pioneer.
    Disclaimer: You are solely responsible for using this app in such a way that it does not impair your ability to drive.
    4
    A coincidence that you ask , but recently I started working on AAUnchained again. I talked to some people about it and got some new insights as how to go forward.
    It's pretty tough but I am slowly making progress. It's pretty stable now when talking to the OpenAuto head unit simulator. I can also talk to my Pioneer head unit via AAwireless but not stable yet. Plan to do some tests to talk to the head unit via USB and iron out the kinks. The code that I have now is basically a prototype and nowhere near production quality.
    4
    AAUnchained

    Thanks to the virus I have now more time available as for my regular job I am now working from home which saves a lot of commuting time.
    So I finally decided to kick off Android Auto Unchained a.k.a AAUnchained a.k.a. AAU. The target is to have one app that impersonates Android Auto, mirrors the phone screen and accepts touch input from the head unit. This without using anything from Android Auto itself. Preferably also without root access.

    Here's my to do list:
    1) Setup simulated Android Auto by using openauto
    2) Get Android Auto head unit
    3) Make Python proof of concept version of AAU
    4) - Get SSL encryption working
    5) - Get video & audio streaming working to mirror the head unit audio & video
    6) - Get touch input from head unit working to inject touch on the phone
    7) Create actual app implementing everything in Java.
    8) Do loads of testing

    Current status:
    1) Done. Have openauto compiled & working on Linux Mint.
    2) Ordered a Pioneer SPH-DA250DAB. I think this is the EU version of the DMH-2600NEX.
    3) Got Python development environment setup on Linux Mint.
    4) Working on it.
    5....) To be done.

    There seems to be one issue that's not easy to solve and that is to capture internal Android audio in order to be able to stream it. This is because Google restricted access to it. I hope there's a way around it or else the app will be pretty useless.