Thanks Austin - so I found the issue here. It wasn't the streams. It was the server. S3 doesn't serve the streams properly I'm not sure the exact header magic to fix it, because I found routing the s3 bucket through the cloudfront CDN would allow them to work, and we're fine with using cloudfront.
Both of these streams work fine off a vanilla Apache instance however.