Welcome to XDA

Search to go directly to your device's forum

Register an account

Unlock full posting privileges

Ask a question

No registration required
Post Reply

Game streaming to other Android devices or PCs with Limelight and GeForce Experience

OP cgutman

31st October 2013, 03:35 PM   |  #1  
OP Senior Member
Thanks Meter: 174
 
309 posts
Join Date:Joined: Aug 2010
More
Android GitHub project: https://github.com/limelight-stream/limelight-android

Video latency increases with GFE 2.1.4
Please read this post.

Android Beta Testing Group
Join the beta testing group to get pre-release builds: https://plus.google.com/communities/...34463232285067

App Store Availability (Non-root version)
Play Store - Latest build (2.9.0.1)
Amazon app store - Recent build (2.5.7.1) - 2.9.0.1 pending
Ouya store - Recent build (2.5.6) - 2.9.0.1 pending

App Store Availability (Root version)
Play Store - Latest build (2.9.0.1)
Amazon app store - Recent build (2.5.7.1) - 2.9.0.1 pending

PC port (beta) - Last updated November 11, 2014
PC port binaries: https://github.com/limelight-stream/...ht-pc/releases
PC port source: https://github.com/limelight-stream/limelight-pc

Streaming non-Steam games/apps
You can stream pretty much anything you can add to your Steam library. There's a tutorial on Nvidia's Shield forum that describes how to do this.

Streaming over the Internet
GeForce Experience 2.0 enables streaming over the Internet. As long as you use the external IP address of your PC and the appropriate ports are forwarded or UPnP is enabled (see http://bit.ly/1e5jDkP for details), you should be able to stream over the Internet.

Ports to forward manually:
TCP 35043, 47984, 47989, 47995, 47996, 48010
UDP 47998, 47999, 48000

Using a wired/wireless controller connected to the GFE PC instead of the streaming device
Normally, Limelight sends controller input from the streaming client which gets sent to the game by GFE. If you want to connect a controller to your PC instead of the streaming device, GFE can cause some problems because the emulated controller still appears to games as controller 1. Luckily there is a workaround for this. You'll need to rename the DLL that Nvidia is using to send controller input so it won't be used anymore. On 32-bit and 64-bit Windows, rename rxinput.dll to rxinput.dll.old on in C:\Program Files\NVIDIA Corporation\NvStreamSrv. On 64-bit versions, there's another copy of the DLL in C:\Program Files (x86)\NVIDIA Corporation\NvStreamSrv which you'll want to rename. You may have to do the renaming again if GFE does an update, but it should allow you to use your controller normally on games that only support 1 controller.

Current Progress
Video and audio are working. Mouse is implemented with support for the touchscreen as a touchpad or using a real mouse or touchpad. Pressing back/select and start together on your controller will bring up the Steam overlay. Keyboard support is also implemented.

General requirements for current APK:
SoC capable of decoding H.264 High Profile in hardware (Snapdragon, Exynos, Tegra 3 or higher, Rockchip, and more) OR an SoC powerful enough to decode in software (4 x Cortex-A9 at 1.5 GHz or similar)
Android 4.1 or higher
GeForce Experience with a GTX 600/700/800 GPU (GTX 600M/700M/800M supported with GeForce Experience 2.0)
Steam
Xbox, PS3 (with SixAxis app), Moga (HID mode), Shield, or Ouya controller (other controllers may work too in HID mode)
Mid to high-end wireless router (preferably dual-band 802.11n or better)
Good wireless connection to your Android device

Troubleshooting tips:
1. Make sure GeForce Experience is installed, up-to-date, and that you've scanned for games.

2. Make sure your device is on the same network as your computer for initial pairing.

3. Try disabling any firewall software running on your machine.

4. Try rebooting your machine. Sometimes the streaming software gets into a messed up state where it won't work normally until the machine is rebooted.

5. Make sure your Android device has a strong wireless connection (and your PC too, if it's connected wirelessly).

6. For Internet streaming, make sure UPnP is enabled on your router or the ports are manually forwarded. See http://bit.ly/1e5jDkP for details.

For those with latency issues, please see this post.

Device-related issues
Depending on the wireless chipset on your phone/tablet, you may have a bad streaming experience if Bluetooth is active while streaming. Unfortunately, there's nothing we can do about this. If you experience significant connection degradation with a Bluetooth controller connected, you could try connecting the controller to your PC (see the section above), a USB Ethernet adapter, or controller that connects directly to your Android device (assuming your Android device supports USB OTG)

Recent Changes:
See GitHub releases page for recent changes: https://github.com/limelight-stream/...droid/releases

Older Changes:

Update 12 - March 13, 2014:
Significant video quality improvements. Lower video latency. New UI that makes it easier to choose the best streaming settings. Transient messages are displayed while streaming if network or device problems are detected.

Update 11:
Tegra hardware decoding latency bug is fixed. Hardware decoding is now used by default on Tegra and Rockchip devices. Performance is vastly improved on Tegra devices (1080p60 decodes in real-time, even on Tegra 3). The parser bug causing additional artifacts and image corruption is (finally) fixed.

Update 10:
Added options to force either hardware or software decoding. Reduce audio decoding CPU usage. Fix image quality and performance regressions from update 9.

Update 9:
Reduced CPU usage of video decoding. Added options to choose target resolution (720p or 1080p) and FPS (30 or 60).

Update 8:
Added a checkbox to choose image quality vs performance (only for CPU decoding). Optimize CPU decoding further. The frame rate is now playable on the Ouya with its Tegra 3..

Update 7:
Connectivity issues should be resolved now. Update to the latest APK if you were experiencing connection failures with the last couple of releases.

Update 6:
There's now GUI feedback when connecting. The whitelist for hardware decoding (that only included Qualcomm decoders) has now been replaced with a blacklist (currently containing TI and Nvidia decoders). The Exynos decoder in Exynos 5 Octa has been confirmed to work.

Update 5:
The app will now request a new reference frame if packet loss occurs on the video stream. This means that the stream will recover from blockiness and artifacting that occur when video packets get lost. CPU decoding for non-Snapdragon devices is a bit better. Fixed back button on Shield.

Update 4:
Added multithreaded CPU H264 decoding support for non-Snapdragon devices with ffmpeg. Both landscape orientations now work. This grows the APK significantly so don't be alarmed when this download is larger than previous builds.

Tegra 4 is now very smooth in the games I've tested. Tegra 3 works significantly better than before, but still not perfect (and won't likely ever be as smooth as Snapdragon or Tegra 4).

For Qualcomm devices, a dual-core SoC (even as old as Cortex-A8 stuff) is sufficient due to the hardware decoder. For other devices, CPU decoding will now be used. These devices will need more CPU horsepower (a quad-core Tegra 3 is almost enough).

Look forward to keyboard support and a better GUI coming in the next several days.

Update 3:
Frame pacing improvements for Snapdragon and Tegra devices, although Tegra still has more latency than Snapdragon devices. If you have issues with blockiness or discoloration in the video stream, make sure that you have a good wireless connection. Limelight doesn't currently deal with packet loss as well as the Shield streaming app.

Update 2:
PS3, Xbox, Shield, and Moga Pro controllers are working with the latest APK.

Update:
Audio is now working. Video is working pretty well on Snapdragon devices (with some lag on Tegra devices). I've attached the current APK here for those that want to test. Due to the framework we're using for video decoding, this app requires Android 4.1 or higher. This is still in alpha so expect bugs.

Original post:
Here is a demo of a WIP app that uses the same Shield streaming technology to stream to any Android device. Controller and mouse input works. Keyboard input isn't implemented yet. Video support works (minus some artifacts at rare points and minor frame pacing issues). Audio doesn't work yet (not sure what format it is).

We've had success with very low H264 decoding latency on Snapdragon S4 Pro/600 devices (like the 2013 Nexus 7 and HTC One), but the Tegra 3/4 decoder has a high latency per frame (~1 second) that makes streaming more laggy on devices like the Ouya, 2012 Nexus 7, and even the Shield itself.

The next big step to a release-ready app is audio support (and the obligatory code cleanup). I'd be happy to respond to any questions about the way the app or the GFE streaming protocol works. If there's significant interest in this, I'll try to put more time into finishing it ASAP.


Demo video (a bit old now):
http://www.youtube.com/watch?v=0VOti83qZRU

Downloads:
I'd recommend downloading the app from the Play Store. Updates are automatically applied through the Play Store when they are released. Crash reports also get to us automatically if you use the Play Store version and click the Report button if Limelight crashes.



For those Amazon folks out there:


Sometimes APKs are more convenient for sideloading and other things, so they will continue to be posted.

You can find the latest APKs on the GitHub page here: https://github.com/limelight-stream/...droid/releases
Last edited by cgutman; 18th November 2014 at 03:48 AM.
The Following 65 Users Say Thank You to cgutman For This Useful Post: [ View ]
1st November 2013, 04:04 AM   |  #2  
Speedy Gonzalez's Avatar
Senior Member
Louisiana
Thanks Meter: 69
 
236 posts
Join Date:Joined: May 2011
More
Quote:
Originally Posted by cgutman

Here is a demo of a WIP app that uses the same Shield streaming technology to stream to any Android device. Controller and mouse input works. Keyboard input isn't implemented yet. Video support works (minus some artifacts at rare points and minor frame pacing issues). Audio doesn't work yet (not sure what format it is).

We've had success with very low H264 decoding latency on Snapdragon S4 Pro/600 devices (like the 2013 Nexus 7 and HTC One), but the Tegra 3/4 decoder has a high latency per frame (~1 second) that makes streaming more laggy on devices like the Ouya, 2012 Nexus 7, and even the Shield itself.

The next big step to a release-ready app is audio support (and the obligatory code cleanup). I'd be happy to respond to any questions about the way the app or the GFE streaming protocol works. If there's significant interest in this, I'll try to put more time into finishing it ASAP.

http://www.youtube.com/watch?v=0VOti83qZRU

sounds very promising you should post this info over the nexus 7 forum HTC and Samsung forums you will get more interest there
I will be happy to try it on my note 3 if you get audio working
The Following User Says Thank You to Speedy Gonzalez For This Useful Post: [ View ]
1st November 2013, 11:41 AM   |  #3  
Senior Member
Thanks Meter: 27
 
290 posts
Join Date:Joined: Mar 2011
very interesting work however I think you might find this

http://forum.xda-developers.com/show....php?t=2506438

a better alternative to work on, maybe if you find a way to get splashtop THD algorithm for streaming, which is leaps and bounds ahead of nvidia's solution. A plus point of this is you can stream anything, your not restricted to just steam and big picture mode.
3rd November 2013, 05:19 PM   |  #4  
Member
Flag Portugal
Thanks Meter: 4
 
36 posts
Join Date:Joined: Nov 2013
More
Really cool!
Tested on Asus TF300 its lagging a lot but i really like what you are doing!
Keep up the good job!


Some questions:
Why no pair button? I had to discover the pair url and do it by hand.
Also why the mac is not read from the android system? Had to change it before compile.

You forgotten to put the link for the code... cant post because new user
Last edited by danielb7390; 3rd November 2013 at 05:24 PM.
4th November 2013, 03:31 AM   |  #5  
OP Senior Member
Thanks Meter: 174
 
309 posts
Join Date:Joined: Aug 2010
More
Quote:
Originally Posted by danielb7390

Really cool!
Tested on Asus TF300 its lagging a lot but i really like what you are doing!
Keep up the good job!


Some questions:
Why no pair button? I had to discover the pair url and do it by hand.
Also why the mac is not read from the android system? Had to change it before compile.

You forgotten to put the link for the code... cant post because new user

The pairing should take place on the first connection. When we get the /pair URL, it should prompt for pairing on the PC. The MAC wasn't read simply because this was some proof of concept code written in a hurry. I'll be fixing that (and other things) shortly.

There's also no UI feedback yet as to what's going on (since we've mostly been debugging it with ADB over the network). I might add some toasts for now to indicate what's going on. In general, it needs cleanup in the UI area along with some code cleanup of some of the early stuff we wrote. There's code in there for mDNS discovery (like the Shield's app uses) which does work but we lack the UI to display the results. We'll also eventually have a proper game selection UI (since we also know how the Shield's app requests game thumbnails), so games can be launched without using Big Picture.

The code link wasn't posted originally because it's a very early proof of concept, but since you asked: https://github.com/cgutman/limelight

EDIT: You were right about the pairing bug. I forgot that we checked if it was paired before attempting to connect. I've added a pairing button and fixed the hardcoded MAC address.
Last edited by cgutman; 4th November 2013 at 07:46 AM. Reason: Development update
The Following User Says Thank You to cgutman For This Useful Post: [ View ]
4th November 2013, 10:52 AM   |  #6  
Member
Flag Portugal
Thanks Meter: 4
 
36 posts
Join Date:Joined: Nov 2013
More
Nice!
"early proof of concept" that works!

I will be following the updates!
5th November 2013, 04:48 PM   |  #7  
Member
Flag Portugal
Thanks Meter: 4
 
36 posts
Join Date:Joined: Nov 2013
More
Stoped working!
Don't know what happened i cant pair...also tried the same way i did yesterday ( manually through firefox) and it doesn't accept!!!
Really don't know whats going on here!!
Last edited by danielb7390; 5th November 2013 at 04:53 PM. Reason: misspell
5th November 2013, 05:10 PM   |  #8  
OP Senior Member
Thanks Meter: 174
 
309 posts
Join Date:Joined: Aug 2010
More
Quote:
Originally Posted by danielb7390

Don't know what happened i cant pair...also tried the same way i did yesterday ( manually through firefox) and it doesn't accept!!!
Really don't know whats going on here!!

Try pairing using the button. It's possible that the MAC address that you specified manually isn't the same one that the code selects now.
The Following User Says Thank You to cgutman For This Useful Post: [ View ]
5th November 2013, 05:27 PM   |  #9  
Member
Flag Portugal
Thanks Meter: 4
 
36 posts
Join Date:Joined: Nov 2013
More
Got it working
Got it working after a pc reboot... stupid windows as always!
Anyways i believe theres a problem in your connection layout i can't click the pair button until i resize "mDNSResultView" because its on top of the buttons or its something else!
Also it's supposed to be working the DNS find? No pc's come up on the list.
I get 1s lag is this normal or i have some problem on my end?
Last edited by danielb7390; 5th November 2013 at 05:46 PM.
5th November 2013, 06:03 PM   |  #10  
OP Senior Member
Thanks Meter: 174
 
309 posts
Join Date:Joined: Aug 2010
More
Quote:
Originally Posted by danielb7390

Got it working after a pc reboot... stupid windows as always!
Anyways i believe theres a problem in your connection layout i can't click the pair button until i resize "mDNSResultView" because its on top of the buttons or its something else!
Also it's supposed to be working the DNS find? No pc's come up on the list.
I get 1s lag is this normal or i have some problem on my end?

The connection layout problem was some code that someone accidentally committed because they didn't diff what they were committing. They've been publicly shamed

The layout is fixed now so the pairing button should work again.

The H264 stream that is being fed to the decoders currently isn't 100% perfect. There are a few issues we're still trying to work out. The Snapdragon decoder seems to be the most lenient and fast. The Tegra decoder seems to be fairly lenient as well, but does have the 1 second lag which is common to both Tegra 3 and Tegra 4 devices. I'm not sure if the decoder is just too slow for real-time decoding or if we're doing something wrong, but I suspect the latter since I think Shield itself uses the hardware H264 decoder when streaming the normal way. Both of these decoders handle our H264 stream better than TI's hardware decoder and Google's software decoder which both crash immediately with "Decoder Failed -2". I'm working on fixing the H264 stream issues in the "av" branch.

The Following User Says Thank You to cgutman For This Useful Post: [ View ]
Post Reply Subscribe to Thread

Tags
android, game streaming, gamestream, geforce experience, shield
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes