{"id":2960,"date":"2015-07-03T11:00:30","date_gmt":"2015-07-03T10:00:30","guid":{"rendered":"http:\/\/www.payneful.co.uk\/blogsplosion\/?p=2960"},"modified":"2018-12-17T09:57:02","modified_gmt":"2018-12-17T09:57:02","slug":"the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it","status":"publish","type":"post","link":"https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/","title":{"rendered":"The PAYNEful Portfolio &#8211; Creating a New YouTube Subscription Feed (or How to Replace Your YouTube Subscription Feed Now Google Has Ditched It)"},"content":{"rendered":"<p><strong>Note:<\/strong> if you&#8217;re looking for code, there&#8217;s a link at the bottom of this article where you can get a copy. This article just explains what the code does.<\/p>\n<p>About a year ago, <a title=\"News Article on ArsTechnica\" href=\"http:\/\/arstechnica.com\/gadgets\/2014\/05\/youtube-shuts-down-public-rss-feeds-of-user-subscriptions\/\" target=\"_blank\" rel=\"noopener\">Google mentioned casually that they would be getting rid of YouTube subscription feeds<\/a><sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-1' id='fnref-2960-1' onclick='return fdfootnote_show(2960)'>1<\/a><\/sup>. Earlier this year they finally made good on their threat and turned the feed off for good. This wasn&#8217;t a big problem for many people who don&#8217;t use RSS in their daily lives<sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-2' id='fnref-2960-2' onclick='return fdfootnote_show(2960)'>2<\/a><\/sup>, but for me it was a damn nuisance as I like all my updates in one place.<\/p>\n<p>Google courteously explained that although the main YouTube feed was going away, <a title=\"Support article on Google for YouTube RSS\" href=\"https:\/\/support.google.com\/youtube\/answer\/6224202\" target=\"_blank\" rel=\"noopener\">you can still access individual channel feeds<\/a>. No problem, right?<\/p>\n<div id=\"attachment_49\" class=\"wp-caption aligncenter\" style=\"width: 409px\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-49 size-full\" title=\"What can I say? I follow a lot of things.\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/06\/OldReader.png\" alt=\"TheOldReader\" width=\"399\" height=\"77\" data-wp-pid=\"49\" \/><div class=\"wp-caption-text\">Hmm.<\/div><\/div>\n<p>My preferred feed reader, <a title=\"I believe the name refers to how it was modelled after Google Reader i.e. &quot;the old one that was scrapped&quot;.\" href=\"https:\/\/theoldreader.com\" target=\"_blank\" rel=\"noopener\">the Old Reader<\/a>, has a limit of up to 100 subscriptions on the free plan, and I have 435 subscriptions on YouTube alone. Please understand that I don&#8217;t have a TV licence as most of my video entertainment comes via YouTube and other online outlets, so it&#8217;s like being subscribed to 400+ channels that I actually want rather than paying Sky or Virgin for 900+ channels that I don&#8217;t want!<\/p>\n<p>I refuse to pay money for extra subscriptions just because Google want to drop a service, so I decided to just build a subscription feed myself. How hard can it be, right?<\/p>\n<h2>Fetching the Feeds<\/h2>\n<p>Google did one good thing at least &#8211; at the bottom of everyone&#8217;s YouTube subscriptions management page is a button allowing you to export all your channel subscriptions to one <a title=\"Outline Processor Markup Language on Wikipedia\" href=\"https:\/\/en.wikipedia.org\/wiki\/OPML\" target=\"_blank\" rel=\"noopener\">OPML<\/a> file. This file would make the basis of my new feed.<\/p>\n<div id=\"attachment_50\" class=\"wp-caption aligncenter\" style=\"width: 1039px\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-50 size-full\" title=\"Why, yes; Doc Brown, Dr. Holocaust AND Doctor Who!\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/06\/YouTubeSubs.png\" alt=\"YouTubeSubs\" width=\"1029\" height=\"356\" data-wp-pid=\"50\" \/><div class=\"wp-caption-text\">In case you&#8217;re playing along at home, here&#8217;s where that fabled button is.<\/div><\/div>\n<p>The file itself just contains data pertaining to where all the individual feeds are and some meta data (titles, channels, etc.).<\/p>\n<div id=\"attachment_51\" class=\"wp-caption aligncenter\" style=\"width: 1360px\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-51 size-full\" title=\"I can't honestly remember what most of these channels are about.\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/06\/YouTubeOPML.png\" alt=\"YouTubeOPML\" width=\"1350\" height=\"324\" data-wp-pid=\"51\" \/><div class=\"wp-caption-text\">It looks like this &#8211; I&#8217;ve actually indented it here to make it a bit more readable as the file comes with one long line of data in it.<\/div><\/div>\n<p>My plan was to have a URL<sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-3' id='fnref-2960-3' onclick='return fdfootnote_show(2960)'>3<\/a><\/sup> that the Old Reader can invoke and pull all the new items from. I know that the Old Reader is clever enough to know which links it has presented before, so all I needed to do was make sure that fresh links within the last day or two were showing and the reader would take care of the rest.<\/p>\n<p>My website is built (mostly) in <a title=\"CodeIgniter, the most lamented of MVC frameworks!\" href=\"http:\/\/www.codeigniter.com\/\">CodeIgniter<\/a>, so I created a new controller class devoted to processing all my feeds. I hard-coded a &#8220;key&#8221; to pass in to prevent people other than my feed reader from accessing my subscriptions, and pointed the script to look at the folder where I would keep my current OPML file. Note that I am hard-coding a lot of things here which isn&#8217;t 100% in the spirit of &#8220;flexible code&#8221;, but it&#8217;s going to serve a specific purpose for me rather than being open to the public so it will do!<\/p>\n<div id=\"attachment_52\" class=\"wp-caption aligncenter\" style=\"width: 886px\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-52 size-full\" title=\"I'm never sure if it's CURL, curl or cURL.\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/06\/YT_Script01.png\" alt=\"YT_Script01\" width=\"876\" height=\"517\" data-wp-pid=\"52\" \/><div class=\"wp-caption-text\">I&#8217;ve blacked out some of the more sensitive areas here so people can&#8217;t start picking my website security apart!<\/div><\/div>\n<p>Unlike the horrendous non-standard markup I encountered when <a title=\"On Parsing Text and HTML with PHP\" href=\"https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/05\/15\/the-payneful-portfolio-on-parsing-text-and-html-with-php\/\">trying to process bookmark files<\/a>, Google&#8217;s OPML is well-structured which means we can use the proper XML parsing tools built into PHP! We load the contents of the file into a <a title=\"SimpleXML in the PHP Manual\" href=\"http:\/\/php.net\/manual\/en\/book.simplexml.php\" target=\"_blank\" rel=\"noopener\">SimpleXML object<\/a> and loop through to collect the two most important items we need per YouTube channel &#8211; the title of the channel and the URL location of where the relevant feed can be found. Note that we have to <a title=\"Tsk. I thought we were trying to get away from type casting.\" href=\"http:\/\/php.net\/manual\/en\/language.types.type-juggling.php\" target=\"_blank\" rel=\"noopener\">&#8220;cast&#8221; the node into a string<\/a> as PHP will try (and fail in) reassigning the object node to a variable otherwise.<\/p>\n<div id=\"attachment_53\" class=\"wp-caption aligncenter\" style=\"width: 1133px\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-53 size-full\" title=\"I do love arrays, yessiree!\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/06\/YT_Script02.png\" alt=\"YouTube Script 02\" width=\"1123\" height=\"573\" data-wp-pid=\"53\" \/><div class=\"wp-caption-text\">It&#8217;s quite fiddly figuring out which nodes to loop through, the best thing I find is to just keep referring back to the source file and indent it out to work out where each node is and what it contains.<\/div><\/div>\n<p>We now have an array of feed URLs. I pass this into a function called _process_feeds() and inside that function we will now need to do the following:<\/p>\n<ul>\n<li>Loop through the array<\/li>\n<li>Request each feed URL and get the contents of the RSS tied to them<\/li>\n<li>Loop through the first two videos on each feed and check the date &#8211; if the most recent videos are older than 2 days we move on to the next feed and do the same<\/li>\n<li>If the videos have been added within the last two days, get all the juicy details and stick them into a results array<\/li>\n<li>Sort the array based on timestamp and return it<\/li>\n<\/ul>\n<p>If _process_feeds() returns an array with some items in it, we have some feed material! The best part was that I <a title=\"PAYNEful Updates RSS feed\" href=\"http:\/\/www.payneful.co.uk\/site\/feed\" target=\"_blank\" rel=\"noopener\">already had an RSS template built<\/a> from when I created an RSS feed that amalgamates some WordPress feeds and a Twitter feed into one RSS. I simply load this existing view and pass in the collected the feed data and hey, presto! One YouTube subscriptions feed.<\/p>\n<h2>The Data Collection Timeout Problem<\/h2>\n<p>It looked good to go, so I uploaded all the files to my web server and plugged the URL into the Old Reader&#8230;<\/p>\n<div id=\"attachment_54\" class=\"wp-caption aligncenter\" style=\"width: 259px\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-54 size-full\" title=\"NOOOOOOO\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/06\/NoFeed.png\" alt=\"No Feed\" width=\"249\" height=\"136\" data-wp-pid=\"54\" \/><div class=\"wp-caption-text\">Well, damn.<\/div><\/div>\n<p>The Old Reader refused to register the URL as valid RSS. After a bit of tinkering, I managed to figure out why: <a title=\"Get Firebug today!\" href=\"http:\/\/getfirebug.com\/\" target=\"_blank\" rel=\"noopener\">Firebug<\/a> revealed that the Old Reader would query a RSS feed for 30 seconds before timing out and presenting an error.<\/p>\n<div id=\"attachment_57\" class=\"wp-caption aligncenter\" style=\"width: 769px\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-57 size-full\" title=\"I'll give you &quot;unprocessable entity&quot;, why I oughta...\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/06\/Feed_Error.png\" alt=\"Feed Error\" width=\"759\" height=\"156\" data-wp-pid=\"57\" \/><div class=\"wp-caption-text\">Firebug is such an invaluable tool: don&#8217;t leave home without it!<\/div><\/div>\n<p>The sheer amount of data was causing my script to take about one minute and thirty seconds to run due to the amount of web calls it was making. Some investigation into optimising the calls suggested <a title=\"CURL vs. file_get_contents()\" href=\"https:\/\/tonydelanuez.com\/PHP-Consuming-REST-APIs-with-CURL-or-file_get_contents\/\" target=\"_blank\" rel=\"noopener\">I should replace PHP&#8217;s file_get_contents() function with CURL<\/a> if it&#8217;s installed on my server. After a quick test to determine I could use CURL, I wrote a small function to get the feeds using CURL and ran the script to see what the difference was.<\/p>\n<div id=\"attachment_58\" class=\"wp-caption aligncenter\" style=\"width: 994px\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-58 size-full\" title=\"Credit where credit's due!\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/06\/CURL.png\" alt=\"CURL\" width=\"984\" height=\"563\" data-wp-pid=\"58\" \/><div class=\"wp-caption-text\">Credit where it&#8217;s due, <a title=\"Using CURL on StackOverflow\" href=\"http:\/\/stackoverflow.com\/questions\/3535799\/file-get-contents-failed-to-open-stream\" target=\"_blank\" rel=\"noopener\">most of this code was lifted from this StackOverflow post<\/a> (I always reference in my scripts when I use someone else&#8217;s code, it&#8217;s just useful when revisiting code later).<\/div><\/div>\n<p>I will be honest, there was no noticeable difference in speed when using CURL. So, without learning a leaner machine language to process the feeds with<sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-4' id='fnref-2960-4' onclick='return fdfootnote_show(2960)'>4<\/a><\/sup>, what could I do?<\/p>\n<h2>Utilising the Power of Cron<\/h2>\n<p>Rather than try and pull all the feed items live from the function, why not just dump out the RSS to a file and present that file&#8217;s address to my feed reader? The feed reader would query the static file and, as long as the contents were regularly updated, it should pick up new items.<\/p>\n<p>A quick Google search revealed that <a title=\"StackOverflow article on loading a CI view into a variable\" href=\"http:\/\/stackoverflow.com\/questions\/11772651\/load-view-into-a-variable\" target=\"_blank\" rel=\"noopener\">I could load the contents of a CodeIgniter view into a variable<\/a>. I would then dump this out into a file somewhere on my server and let the feed reader do the rest.<\/p>\n<div id=\"attachment_60\" class=\"wp-caption aligncenter\" style=\"width: 643px\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-60 size-full\" title=\"Alright, I admit it, my function names aren't particularly imaginative.\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/06\/File_Write.png\" alt=\"File Write\" width=\"633\" height=\"344\" data-wp-pid=\"60\" \/><div class=\"wp-caption-text\">I&#8217;ve used <a title=\"file_put_contents() in the PHP manual\" href=\"http:\/\/php.net\/manual\/en\/function.file-put-contents.php\" target=\"_blank\" rel=\"noopener\">file_put_contents()<\/a> before, but I&#8217;m pleased to report that fopen and fwrite are a lot easier to work with!<\/div><\/div>\n<p>I then set up a cron job to run the main process_feeds() function once every six hours to refresh the contents of my RSS file, which is fairly simple as CodeIgniter has rudimentary <a title=\"CLI in the CodeIgniter manual\" href=\"https:\/\/ellislab.com\/codeigniter\/user-guide\/general\/cli.html\" target=\"_blank\" rel=\"noopener\">CLI functionality<\/a><sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-5' id='fnref-2960-5' onclick='return fdfootnote_show(2960)'>5<\/a><\/sup>.<\/p>\n<p>As an aside, for some bizarre reason my hosting provider&#8217;s cron panel just didn&#8217;t like the supposed CodeIgniter CLI invocation format at all. A line like this&#8230;<\/p>\n<pre>0 *\/6 * * * php -q [FILE PATH]\/index.php youtube fetch_feed [KEY] &gt;\/dev\/null 2&gt;&amp;1<\/pre>\n<p>&#8230;that should have worked fine just returned &#8220;content type: text\/html&#8221; and refused to output anything else. It ran fine on a Linux box via the terminal and I spent a lot of time commenting out bits and pieces to see if it was my unique setup of CodeIgniter integrated with WordPress causing the problem. In the end, <a title=\"6 CodeIgniter Hacks for the Masters\" href=\"http:\/\/code.tutsplus.com\/tutorials\/6-codeigniter-hacks-for-the-masters--net-8308\" target=\"_blank\" rel=\"noopener\">I found this excellent article<\/a> and set up a separate cli.php file which worked just fine, invoked with the following cron:<\/p>\n<pre>0 *\/6 * * * php -q [FILE PATH]\/cli.php \"youtube\/fetch_feed\/[KEY]\" &gt;\/dev\/null 2&gt;&amp;1<\/pre>\n<p>If you can get away with using CodeIgniter&#8217;s &#8220;built-in&#8221; CLI interface, then bully for you!<\/p>\n<h2>The Static File Problem<\/h2>\n<p>To recap, I now have a static .rss file that updates every six hours with new videos from the last two days. Yet, there&#8217;s a problem. Beyond the initial import, the Old Reader refuses to register any new videos. It is particularly frustrating since at this point I am regularly checking the .rss file and it definitely has new videos showing in it.<\/p>\n<p>After some research, it appears that my previous assumption<sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-6' id='fnref-2960-6' onclick='return fdfootnote_show(2960)'>6<\/a><\/sup> that feed readers would just scan for RSS nodes with a recent date was incorrect. On reflection, it&#8217;s a little bit obvious: RSS feed aggregators could scan feeds for certain dates based on the last update date, but that&#8217;s a lot of processing to fetch a feed and check the videos, only to find there are no new ones.<\/p>\n<p>No, instead a lot of services will apparently send a HTTP<sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-7' id='fnref-2960-7' onclick='return fdfootnote_show(2960)'>7<\/a><\/sup> request and <a title=\"The Fishbowl on headers for RSS\" href=\"http:\/\/fishbowl.pastiche.org\/2002\/10\/21\/http_conditional_get_for_rss_hackers\/\" target=\"_blank\" rel=\"noopener\">if the page returns specific headers as to when the page was last updated it will then poll for new content<\/a>.<\/p>\n<p>I&#8217;ll admit it now, my understanding of headers is fuzzy at best. I know it is invisible information passed to a server before a web page request, <a title=\"HTTP codes via W3C\" href=\"http:\/\/www.w3.org\/Protocols\/rfc2616\/rfc2616-sec10.html\" target=\"_blank\" rel=\"noopener\">normally replying in the form of a response code<\/a>, and that it can contain a lot of information pertaining to how a page should behave; beyond that, it is currently a bit of a mystery to me. I have previously set headers for a page when I wanted certain files to be prompted as a &#8220;download now&#8221; (see the <a title=\"On Parsing Text and HTML with PHP\" href=\"https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/05\/15\/the-payneful-portfolio-on-parsing-text-and-html-with-php\/\">HTML bookmark sorter article<\/a> for an example), when redirecting as <a title=\"The amount of errors I've had trouble with when invoking a PHP redirect because &quot;they have already been sent&quot;...\" href=\"http:\/\/php.net\/manual\/en\/function.header.php\" target=\"_blank\" rel=\"noopener\">PHP can send a redirect condition to go to a new page<\/a>, or even something as simple as <a title=\"RIP Sir Terry\" href=\"http:\/\/www.payneful.co.uk\/blogsplosion\/2015\/03\/14\/ode-to-a-beautiful-wordsmith-rip-sir-terry-pratchett\/\" target=\"_blank\" rel=\"noopener\">a gesture towards your favourite recently-deceased author<\/a> by <a title=\"RIP Sir Terry\" href=\"http:\/\/www.gnuterrypratchett.com\/\" target=\"_blank\" rel=\"noopener\">placing his name in the headers of your website<\/a> via .htaccess<sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-8' id='fnref-2960-8' onclick='return fdfootnote_show(2960)'>8<\/a><\/sup>.<\/p>\n<p>I wrote a new function called show_feed() as part of the class, which returns the contents of the RSS file I generated and prefixes it with a header providing the date of the latest video in the feed. Then I just output the contents of the RSS feed. It was a bit trial and error and I had to use a few online resources to fathom it out, which I have referenced in the final script.<\/p>\n<div id=\"attachment_64\" class=\"wp-caption aligncenter\" style=\"width: 1040px\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-64 size-full\" title=\"show_feed() Function\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/07\/show_feed.png\" alt=\"show_feed() Function\" width=\"1030\" height=\"691\" data-wp-pid=\"64\" \/><div class=\"wp-caption-text\">I also defined a bunch of variables at the top of the script for convenience, have all the &#8220;$this-&gt;&#8221; references!<\/div><\/div>\n<p>At first my feed reader did not want to update at all. Then I forced a refresh and then, lo and behold, on the following day I found the following in my feed reader&#8230;<\/p>\n<div id=\"attachment_63\" class=\"wp-caption aligncenter\" style=\"width: 1376px\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-63\" title=\"Old Reader Feed\" src=\"http:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/2015\/07\/OldReader_Feed.png\" alt=\"Old Reader Feed\" width=\"1366\" height=\"605\" data-wp-pid=\"63\" \/><div class=\"wp-caption-text\"><a title=\"YES! YES! YES! I spent about twenty hours on this smegging project.\" href=\"https:\/\/www.youtube.com\/watch?v=u6_nJ11BgTE\" target=\"_blank\" rel=\"noopener\">HALLELUJAH!<\/a><\/div><\/div>\n<h2>Potential Improvements and the Code Base<\/h2>\n<p>There are a few benefits to my feed script:<\/p>\n<ul>\n<li>The YouTube subscriptions feed used to have a massive image and no author in the title, so it looked ugly as hell. Every video in my feed script prefixes the video and description with the channel name for quick reference, and the image is now thumbnail-sized on the right.<\/li>\n<li>At some point towards the end of the YouTube subscription&#8217;s lifespan <a title=\"OMG people aren't going to get their PewDiePie videos! Thank Christ.\" href=\"https:\/\/www.youtube.com\/watch?v=HLJQ0gFHM8s\" target=\"_blank\" rel=\"noopener\">there were rumours of favouritism in what video were shown<\/a> i.e. if you tended to click on videos from certain channels, the feed would then favour those channels over others<sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-9' id='fnref-2960-9' onclick='return fdfootnote_show(2960)'>9<\/a><\/sup>. I suspect this was to lower the amount of unnecessary processing, as developing this script has taught me that collecting and collating information from so many web resources is quite an intensive process &#8211; I can only imagine what the resource usage was for the Google team in providing RSS subscription feeds for 1 billion+ people<sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-10' id='fnref-2960-10' onclick='return fdfootnote_show(2960)'>10<\/a><\/sup>. My feed script has no favouritism at all, it just collects videos: if you get fed up of a channel, just remove them from the .OPML file!<\/li>\n<\/ul>\n<p>However, there are also quite a few flaws in this script:<\/p>\n<ul>\n<li>The script only checks two videos per feed<sup class='footnote'><a href='https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/#fn-2960-11' id='fnref-2960-11' onclick='return fdfootnote_show(2960)'>11<\/a><\/sup> to keep the processing time down &#8211; if a channel uploads five videos in a short amount of time you&#8217;ll probably miss a few of them! There&#8217;s a variable called &#8220;vid_limit&#8221; that can be adjusted to increase this amount, but be aware that it will increase processing time.<\/li>\n<li>If you have a lot of subscriptions the script will take a while to process them. Most of the bottleneck happens around all the web calls it makes. I can certainly understand why Google were keen to stop offering this sort of feed service if it was half as intense on their servers!<\/li>\n<li>There&#8217;s a lot of hard coding here and reliance on a static file being in a certain location &#8211; the script could certainly be adapted to utilise a database to store the feed locations and then have some sort of interface for adding YT channels to it.<\/li>\n<\/ul>\n<p>I would love to be able to offer this YouTube subscription feed service on my website (like <a title=\"HTML Bookmark Sorter\" href=\"http:\/\/www.payneful.co.uk\/projects\/bookmark_sorter\">the bookmark sorter<\/a>) but it might prove costly in terms of bandwidth. Instead, <a title=\"Code hosted on GitHub\" href=\"https:\/\/github.com\/SPPayne\/ci_youtube_subs_feed_amalgamator\" target=\"_blank\" rel=\"noopener\">I have placed all the code on GitHub into a repository<\/a>. Please help yourself to it if it is of any value to you. It&#8217;s been designed to run in CodeIgniter but I imagine it would be quite simple to convert it to another MVC framework or just as a series of procedural static functions.<\/p>\n<p><a title=\"Code hosted on GitHub\" href=\"https:\/\/github.com\/SPPayne\/ci_youtube_subs_feed_amalgamator\" target=\"_blank\" rel=\"noopener\">Get the code for the YouTube Subscription Amalgamator here<\/a>.<\/p>\n<p>If you have any suggestions on how to improve my script, please let me know with a comment below (you can login using most popular social media accounts) or get in touch on GitHub (in which case please bear with me while I learn the process &#8211; I have used GitHub for versioning but not so much for collaboration!).<\/p>\n<h2>Postscript (16\/07\/2015)<\/h2>\n<p>This doesn&#8217;t have anything to do with the subscription feed, but I updated my CodeIgniter framework to 3.0 this week and, lo and behold, the script stopped running via the command line. My &#8220;hacked&#8221; cli.php file didn&#8217;t work, and using other methods proved fruitless. It seems that <a title=\"JustHost! Does what it says on the tin. Command line? That&#039;s not hosting!\" href=\"https:\/\/web.archive.org\/web\/20151207224253\/http:\/\/activeden.net:80\/forums\/thread\/cron-jobs-on-justhostcom\/37562\" target=\"_blank\" rel=\"noopener\">a few people have the same problem with using CI via CLI on my hosting provider<\/a>.<\/p>\n<p>Luckily, I found a forum post where <a title=\"GENIUS!\" href=\"https:\/\/web.archive.org\/web\/20151207224253\/http:\/\/activeden.net:80\/forums\/thread\/cron-jobs-on-justhostcom\/37562\" target=\"_blank\" rel=\"noopener\">a clever sod named &#8220;ZoomIt&#8221; suggested using Wget instead<\/a>. I&#8217;m just trying to load the controller function fetch_feed(), after all! I added a modifier to <a title=\"Suppressing wget output\" href=\"http:\/\/stackoverflow.com\/questions\/9691367\/how-do-i-request-a-file-but-not-save-it-with-wget\" target=\"_blank\" rel=\"noopener\">suppress the output<\/a> (dump it out to the black hole of \/dev\/null) and hey, presto! One working feed generator, and this cron issue didn&#8217;t take a week to resolve like last time. The resulting cronjob looked like the following:<\/p>\n<pre>wget -qO- http:\/\/www.payneful.co.uk\/youtube\/fetch_feed\/[KEY] &amp;&gt; \/dev\/null<\/pre>\n<p>It&#8217;s not as neat as invoking the PHP on the command line and a few forum posts recommended CURL instead as <a title=\"Wget on Wikipedia\" href=\"https:\/\/en.wikipedia.org\/wiki\/Wget\" target=\"_blank\" rel=\"noopener\">Wget is more for copying file output<\/a>, but Wget works so I&#8217;m happy again.<\/p>\n<hr \/>\n","protected":false},"excerpt":{"rendered":"<a href=\"https:\/\/www.payneful.co.uk\/blogsplosion\/2015\/07\/03\/the-payneful-portfolio-creating-a-new-youtube-subscription-feed-or-how-to-replace-your-youtube-subscription-feed-now-google-has-ditched-it\/\"><img width=\"125\" height=\"117\" src=\"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-content\/uploads\/YouTubeUnhappy-150x140.png\" class=\"alignright tfe wp-post-image\" alt=\"YouTube RSS\" decoding=\"async\" loading=\"lazy\" \/><\/a><p>Note: if you&#8217;re looking for code, there&#8217;s a link at the bottom of this article where you can get a copy. This article just explains what the code does. About a year ago, Google mentioned casually that they would be getting rid of YouTube subscription feeds1. Earlier this year they&#8230;<\/p>\n","protected":false},"author":1,"featured_media":2961,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[383,382,308,387,37],"tags":[393,390,392,394,395,396],"class_list":["post-2960","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-articles","category-portfolio-and-work","category-projects","category-projects-portfolio-and-work","category-technology","tag-google","tag-php","tag-please-dont-hate-me-for-my-horrible-code","tag-rss","tag-xml","tag-youtube"],"_links":{"self":[{"href":"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-json\/wp\/v2\/posts\/2960","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-json\/wp\/v2\/comments?post=2960"}],"version-history":[{"count":0,"href":"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-json\/wp\/v2\/posts\/2960\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-json\/wp\/v2\/media\/2961"}],"wp:attachment":[{"href":"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-json\/wp\/v2\/media?parent=2960"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-json\/wp\/v2\/categories?post=2960"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.payneful.co.uk\/blogsplosion\/wp-json\/wp\/v2\/tags?post=2960"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}