5,603,729 Members 37,282 Now Online
XDA Developers Android and Mobile Development Forum

Game streaming to other Android devices or PCs with Limelight and GeForce Experience

Tip us?
 
cgutman
Old
(Last edited by cgutman; 18th April 2014 at 02:22 AM.)
#1  
Senior Member - OP
Thanks Meter 103
Posts: 183
Join Date: Aug 2010
Default Game streaming to other Android devices or PCs with Limelight and GeForce Experience

Android GitHub project: https://github.com/limelight-stream/limelight-android

PC port (beta) now available - Updated March 13, 2014
PC port binaries: https://github.com/limelight-stream/...ht-pc/releases
PC port source: https://github.com/limelight-stream/limelight-pc
See http://forum.xda-developers.com/show...&postcount=170 for more details

Streaming non-Steam games/apps
You can stream pretty much anything you can add to your Steam library. There's a tutorial on Nvidia's Shield forum that describes how to do this.

Streaming over the Internet
GeForce Experience 2.0 enables streaming over the Internet. As long as you use the external IP address of your PC and the appropriate ports are forwarded or UPnP is enabled (see http://bit.ly/1e5jDkP for details), you should be able to stream over the Internet.

Ports to forward manually:
TCP 35043, 47984, 47989, 47991, 47995, 47996
UDP 47998, 47999, 48000

Using a wired/wireless controller connected to the GFE PC instead of the streaming device
Normally, Limelight sends controller input from the streaming client which gets sent to the game by GFE. If you want to connect a controller to your PC instead of the streaming device, GFE can cause some problems because the emulated controller still appears to games as controller 1. Luckily there is a workaround for this. You'll need to rename the DLL that Nvidia is using to send controller input so it won't be used anymore. On 32-bit and 64-bit Windows, rename rxinput.dll to rxinput.dll.old on in C:\Program Files\NVIDIA Corporation\NvStreamSrv. On 64-bit versions, there's another copy of the DLL in C:\Program Files (x86)\NVIDIA Corporation\NvStreamSrv which you'll want to rename. You may have to do the renaming again if GFE does an update, but it should allow you to use your controller normally on games that only support 1 controller.

Current Progress
Video and audio are working. Mouse is implemented with support for the touchscreen as a touchpad or using a real mouse or touchpad. Pressing back/select and start together on your controller will bring up the Steam overlay. Keyboard support is also implemented.

General requirements for current APK:
SoC capable of decoding H.264 High Profile in hardware (Snapdragon, Exynos, Tegra 3 or higher, Rockchip, and more) OR an SoC powerful enough to decode in software (4 x Cortex-A9 at 1.5 GHz or similar)
Android 4.1 or higher
GeForce Experience with a GTX 600/700/800 GPU (GTX 600M/700M/800M supported with GeForce Experience 2.0)
Steam enrolled in beta updates
Xbox, PS3 (with SixAxis app), Moga Pro (HID mode), Shield, or Ouya controller (other controllers may work too in HID mode)
Mid to high-end wireless router (preferably dual-band 802.11n or better)
Good wireless connection to your Android device

Troubleshooting tips:
1. Make sure Steam is installed, updated, and enrolled in beta updates.

2. Make sure GeForce Experience is installed, up-to-date, and that you've scanned for games. If you get a "Steam not found in app list" error when trying to stream, make sure that you've added your steam directory in GFE.

3. Make sure your device is on the same network as your computer and that you're using the correct IP address (the IP of the computer on the network, not what you get from whatsmyip.org unless doing streaming over the Internet, etc)

4. Try disabling any firewall software running on your machine.

5. Try rebooting your machine. Sometimes the streaming software gets into a messed up state where it won't work normally until the machine is rebooted.

6. Make sure your Android device has a strong wireless connection (and your PC too, if it's connected wirelessly).

7. For Internet streaming, make sure UPnP is enabled on your router or the ports are manually forwarded. See http://bit.ly/1e5jDkP for details.

For those with latency issues, please see this post.
If you have an "Steam not found in GFE app list" error, see this post.

Device-related issues
Users with Samsung Galaxy S3 phones running Android 4.3 experiencing crashes when trying to stream should set the "Force Software Decoding" option. There appears to be a bug in the update which causes the hardware decoding code to crash. As far as I can tell, it's not our fault but there may be some workaround we can implement so hardware decoding can be used.

Depending on the wireless chipset on your phone/tablet, you may have a bad streaming experience if Bluetooth is active while streaming. Unfortunately, there's nothing we can do about this. If you experience significant connection degradation with a Bluetooth controller connected, you could try connecting the controller to your PC (see the section above), a USB Ethernet adapter, or controller that connects directly to your Android device (assuming your Android device supports USB OTG)

Recent Changes:
See GitHub releases page for recent changes: https://github.com/limelight-stream/...droid/releases

Older Changes:
 

Update 12 - March 13, 2014:
Significant video quality improvements. Lower video latency. New UI that makes it easier to choose the best streaming settings. Transient messages are displayed while streaming if network or device problems are detected.

Update 11:
Tegra hardware decoding latency bug is fixed. Hardware decoding is now used by default on Tegra and Rockchip devices. Performance is vastly improved on Tegra devices (1080p60 decodes in real-time, even on Tegra 3). The parser bug causing additional artifacts and image corruption is (finally) fixed.

Update 10:
Added options to force either hardware or software decoding. Reduce audio decoding CPU usage. Fix image quality and performance regressions from update 9.

Update 9:
Reduced CPU usage of video decoding. Added options to choose target resolution (720p or 1080p) and FPS (30 or 60).

Update 8:
Added a checkbox to choose image quality vs performance (only for CPU decoding). Optimize CPU decoding further. The frame rate is now playable on the Ouya with its Tegra 3..

Update 7:
Connectivity issues should be resolved now. Update to the latest APK if you were experiencing connection failures with the last couple of releases.

Update 6:
There's now GUI feedback when connecting. The whitelist for hardware decoding (that only included Qualcomm decoders) has now been replaced with a blacklist (currently containing TI and Nvidia decoders). The Exynos decoder in Exynos 5 Octa has been confirmed to work.

Update 5:
The app will now request a new reference frame if packet loss occurs on the video stream. This means that the stream will recover from blockiness and artifacting that occur when video packets get lost. CPU decoding for non-Snapdragon devices is a bit better. Fixed back button on Shield.

Update 4:
Added multithreaded CPU H264 decoding support for non-Snapdragon devices with ffmpeg. Both landscape orientations now work. This grows the APK significantly so don't be alarmed when this download is larger than previous builds.

Tegra 4 is now very smooth in the games I've tested. Tegra 3 works significantly better than before, but still not perfect (and won't likely ever be as smooth as Snapdragon or Tegra 4).

For Qualcomm devices, a dual-core SoC (even as old as Cortex-A8 stuff) is sufficient due to the hardware decoder. For other devices, CPU decoding will now be used. These devices will need more CPU horsepower (a quad-core Tegra 3 is almost enough).

Look forward to keyboard support and a better GUI coming in the next several days.

Update 3:
Frame pacing improvements for Snapdragon and Tegra devices, although Tegra still has more latency than Snapdragon devices. If you have issues with blockiness or discoloration in the video stream, make sure that you have a good wireless connection. Limelight doesn't currently deal with packet loss as well as the Shield streaming app.

Update 2:
PS3, Xbox, Shield, and Moga Pro controllers are working with the latest APK.

Update:
Audio is now working. Video is working pretty well on Snapdragon devices (with some lag on Tegra devices). I've attached the current APK here for those that want to test. Due to the framework we're using for video decoding, this app requires Android 4.1 or higher. This is still in alpha so expect bugs.

Original post:
Here is a demo of a WIP app that uses the same Shield streaming technology to stream to any Android device. Controller and mouse input works. Keyboard input isn't implemented yet. Video support works (minus some artifacts at rare points and minor frame pacing issues). Audio doesn't work yet (not sure what format it is).

We've had success with very low H264 decoding latency on Snapdragon S4 Pro/600 devices (like the 2013 Nexus 7 and HTC One), but the Tegra 3/4 decoder has a high latency per frame (~1 second) that makes streaming more laggy on devices like the Ouya, 2012 Nexus 7, and even the Shield itself.

The next big step to a release-ready app is audio support (and the obligatory code cleanup). I'd be happy to respond to any questions about the way the app or the GFE streaming protocol works. If there's significant interest in this, I'll try to put more time into finishing it ASAP.


Demo video (a bit old now):
http://www.youtube.com/watch?v=0VOti83qZRU

Downloads:
I'd recommend downloading the app from the Play Store. Updates are automatically applied through the Play Store when they are released. Crash reports also get to us automatically if you use the Play Store version and click the Report button if Limelight crashes.



Sometimes APKs are more convenient for sideloading and other things, so they will continue to be posted.

You can find the latest APKs on the GitHub page here: https://github.com/limelight-stream/...droid/releases
The Following 47 Users Say Thank You to cgutman For This Useful Post: [ Click to Expand ]
 
Speedy Gonzalez
Old
#2  
Speedy Gonzalez's Avatar
Senior Member
Thanks Meter 68
Posts: 230
Join Date: May 2011
Location: Louisiana
Quote:
Originally Posted by cgutman View Post
Here is a demo of a WIP app that uses the same Shield streaming technology to stream to any Android device. Controller and mouse input works. Keyboard input isn't implemented yet. Video support works (minus some artifacts at rare points and minor frame pacing issues). Audio doesn't work yet (not sure what format it is).

We've had success with very low H264 decoding latency on Snapdragon S4 Pro/600 devices (like the 2013 Nexus 7 and HTC One), but the Tegra 3/4 decoder has a high latency per frame (~1 second) that makes streaming more laggy on devices like the Ouya, 2012 Nexus 7, and even the Shield itself.

The next big step to a release-ready app is audio support (and the obligatory code cleanup). I'd be happy to respond to any questions about the way the app or the GFE streaming protocol works. If there's significant interest in this, I'll try to put more time into finishing it ASAP.

http://www.youtube.com/watch?v=0VOti83qZRU
sounds very promising you should post this info over the nexus 7 forum HTC and Samsung forums you will get more interest there
I will be happy to try it on my note 3 if you get audio working
Samsung Galaxy note 3 VZW Rooted
iPad mini r Jailbroken
 
nielo360
Old
#3  
Senior Member
Thanks Meter 19
Posts: 235
Join Date: Mar 2011
very interesting work however I think you might find this

http://forum.xda-developers.com/show....php?t=2506438

a better alternative to work on, maybe if you find a way to get splashtop THD algorithm for streaming, which is leaps and bounds ahead of nvidia's solution. A plus point of this is you can stream anything, your not restricted to just steam and big picture mode.
 
danielb7390
Old
(Last edited by danielb7390; 3rd November 2013 at 04:24 PM.)
#4  
Member
Thanks Meter 4
Posts: 35
Join Date: Nov 2013
Location: Portugal
Really cool!
Tested on Asus TF300 its lagging a lot but i really like what you are doing!
Keep up the good job!


Some questions:
Why no pair button? I had to discover the pair url and do it by hand.
Also why the mac is not read from the android system? Had to change it before compile.

You forgotten to put the link for the code... cant post because new user
 
cgutman
Old
(Last edited by cgutman; 4th November 2013 at 06:46 AM.) Reason: Development update
#5  
Senior Member - OP
Thanks Meter 103
Posts: 183
Join Date: Aug 2010
Quote:
Originally Posted by danielb7390 View Post
Really cool!
Tested on Asus TF300 its lagging a lot but i really like what you are doing!
Keep up the good job!


Some questions:
Why no pair button? I had to discover the pair url and do it by hand.
Also why the mac is not read from the android system? Had to change it before compile.

You forgotten to put the link for the code... cant post because new user
The pairing should take place on the first connection. When we get the /pair URL, it should prompt for pairing on the PC. The MAC wasn't read simply because this was some proof of concept code written in a hurry. I'll be fixing that (and other things) shortly.

There's also no UI feedback yet as to what's going on (since we've mostly been debugging it with ADB over the network). I might add some toasts for now to indicate what's going on. In general, it needs cleanup in the UI area along with some code cleanup of some of the early stuff we wrote. There's code in there for mDNS discovery (like the Shield's app uses) which does work but we lack the UI to display the results. We'll also eventually have a proper game selection UI (since we also know how the Shield's app requests game thumbnails), so games can be launched without using Big Picture.

The code link wasn't posted originally because it's a very early proof of concept, but since you asked: https://github.com/cgutman/limelight

EDIT: You were right about the pairing bug. I forgot that we checked if it was paired before attempting to connect. I've added a pairing button and fixed the hardcoded MAC address.
The Following User Says Thank You to cgutman For This Useful Post: [ Click to Expand ]
 
danielb7390
Old
#6  
Member
Thanks Meter 4
Posts: 35
Join Date: Nov 2013
Location: Portugal
Nice!
"early proof of concept" that works!

I will be following the updates!
 
danielb7390
Old
(Last edited by danielb7390; 5th November 2013 at 03:53 PM.) Reason: misspell
#7  
Member
Thanks Meter 4
Posts: 35
Join Date: Nov 2013
Location: Portugal
Default Stoped working!

Don't know what happened i cant pair...also tried the same way i did yesterday ( manually through firefox) and it doesn't accept!!!
Really don't know whats going on here!!
 
cgutman
Old
#8  
Senior Member - OP
Thanks Meter 103
Posts: 183
Join Date: Aug 2010
Quote:
Originally Posted by danielb7390 View Post
Don't know what happened i cant pair...also tried the same way i did yesterday ( manually through firefox) and it doesn't accept!!!
Really don't know whats going on here!!
Try pairing using the button. It's possible that the MAC address that you specified manually isn't the same one that the code selects now.
The Following User Says Thank You to cgutman For This Useful Post: [ Click to Expand ]
 
danielb7390
Old
(Last edited by danielb7390; 5th November 2013 at 04:46 PM.)
#9  
Member
Thanks Meter 4
Posts: 35
Join Date: Nov 2013
Location: Portugal
Default Got it working

Got it working after a pc reboot... stupid windows as always!
Anyways i believe theres a problem in your connection layout i can't click the pair button until i resize "mDNSResultView" because its on top of the buttons or its something else!
Also it's supposed to be working the DNS find? No pc's come up on the list.
I get 1s lag is this normal or i have some problem on my end?
 
cgutman
Old
#10  
Senior Member - OP
Thanks Meter 103
Posts: 183
Join Date: Aug 2010
Quote:
Originally Posted by danielb7390 View Post
Got it working after a pc reboot... stupid windows as always!
Anyways i believe theres a problem in your connection layout i can't click the pair button until i resize "mDNSResultView" because its on top of the buttons or its something else!
Also it's supposed to be working the DNS find? No pc's come up on the list.
I get 1s lag is this normal or i have some problem on my end?
The connection layout problem was some code that someone accidentally committed because they didn't diff what they were committing. They've been publicly shamed

The layout is fixed now so the pairing button should work again.

The H264 stream that is being fed to the decoders currently isn't 100% perfect. There are a few issues we're still trying to work out. The Snapdragon decoder seems to be the most lenient and fast. The Tegra decoder seems to be fairly lenient as well, but does have the 1 second lag which is common to both Tegra 3 and Tegra 4 devices. I'm not sure if the decoder is just too slow for real-time decoding or if we're doing something wrong, but I suspect the latter since I think Shield itself uses the hardware H264 decoder when streaming the normal way. Both of these decoders handle our H264 stream better than TI's hardware decoder and Google's software decoder which both crash immediately with "Decoder Failed -2". I'm working on fixing the H264 stream issues in the "av" branch.

The Following User Says Thank You to cgutman For This Useful Post: [ Click to Expand ]
Tags
android, game streaming, gamestream, geforce experience, shield
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes