Send RSS feed to Webhook?

Troubleshooting
1

I thought it should be possible to fetch RSS feeds via cronjob(s) and send them to my Webhook into an channel. Unfortunately i got stuck after a hour or two trying my best with curl, sed, grep and others.

At the end I figured out that rsstail could be a part of the solution, but I don´t know how exactly I can redirect it to my Webhook.

Is there anyone who can help me out?

Thanks in advance,
Markus

2

Hi @elpatron68,

Would love to see you get this working, could you let us know more about where you got stuck?

Also, there’s some updated documentation and we’re working on a sample to share with the release of v1.1 next Friday.

If you run a curl command from terminal, are you able to post to your Mattermost instance? e.g.:

 curl -i -X POST -d 'payload={"text": "Hello, this is some text."}' http://yourmattermost.com/hooks/xxx-generatedkey-xxx
3

Hello @it33,

sorry, my linux shell experience is a little bit clumsy. Sometimes i find my solution by trial and errror, sometimes not. I hoped, this problem would have been solved by someone else yet.

My approach was using curl to fetch rss feeds and to send them to my (yet working) webhook. I started at http://www.askdavetaylor.com/can_i_track_an_rss_feed_with_a_shell_script/, got stuck and stumbled upon rsstail, which could lead to a solution, but I have no idea how to solve this.

Bye,
Markus

4

Hi @it33,

i answered on Sunday, but Akismet rated my reply as spam.

5

Thanks @elpatron68,

Sorry, didn’t see the spam post until you mentioned it.

I’d propose dividing the issue into parts. First, can we have your help following the steps for testing a Mattermost webhook and let us know the result?

After we confirm the webhook is working we can go to the next step around RSS

6

Hello @it33,

Thank you.

I enabled incoming webhooks, added a new webhook to a channel and successfully manged to send a ‘Hello world’ with curl. So far there were no problems.

7

Excellent, thanks @elpatron68,

I think the next step would be seeing if the cron job you have to pull RSS feeds can post with the curl command,

Please let us know how it goes?

Also, after it’s working, would you be open to posting the cron job here so other community members have a sample to start with?

8

Okay. The shell is not my best friend, so I tried my luck with Python.

This is what I´ve got til now:

#!/usr/bin/python
# -*- coding: utf-8 -*-

import feedparser
import pycurl
import json

# feed_name = sys.argv[1]
# url = sys.argv[2]

feed_name = 'Heise Online News'
feed_url = 'http://heise.de.feedsportal.com/c/35207/f/653902/index.rss'
webhook = 'http://mattermost.medisoft.local:8065/hooks/cns14rjjfpfrxpgoujxpm7sy3w'

d = feedparser.parse(feed_url)
title = d['entries'][0]['title']
link = d['entries'][0]['link']

# This works so far...
print(title)
print(link)

# Example works fine:
# curl -i -X POST -d 'payload={"text": "Hello world."}' http://mattermost.medisoft.local:8065/hooks/cns14rjjfpfrxpgoujxpm7sy3w

c = pycurl.Curl()
payload = '{"channel": "testing", "text": "Hello, this is some text."}'
post = json.dumps(payload)

c.setopt(pycurl.URL, webhook)
c.setopt(pycurl.HTTPHEADER, ['Accept: application/json', 'Content-Type: application/json'])
c.setopt(pycurl.VERBOSE, 1)
c.setopt(pycurl.POST, 1)
c.setopt(pycurl.POSTFIELDS, post)
c.perform()

Unfortunately this results in

* Connected to mattermost.medisoft.local (192.168.0.238) port 8065 (#0)
> POST /hooks/cns14rjjfpfrxpgoujxpm7sy3w HTTP/1.1
User-Agent: PycURL/7.19.5.1 libcurl/7.37.0 WinSSL zlib/1.2.8
Host: mattermost.medisoft.local:8065
Accept: application/json
Content-Type: application/json
Content-Length: 77

* upload completely sent off: 77 out of 77 bytes
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< X-Ratelimit-Limit: 10
< X-Ratelimit-Remaining: 9
< X-Ratelimit-Reset: 1
< X-Request-Id: 1ng8bjjiw3rifcq1eo19cuf51r
< X-Version-Id: 1.0.0.1444332854
< Date: Wed, 14 Oct 2015 16:43:01 GMT
< Content-Length: 128
< 
* Connection #0 to host mattermost.medisoft.local left intact
{"message":"No text specified","detailed_error":"","request_id":"1ng8bjjiw3rifcq1eo19cuf51r","status_code":500,"is_oauth":false}
Process finished with exit code 0
9

Thanks @elpatron68, could you print out the curl command prepared prior to c.perform() and just see how it looks, maybe try it manually?

10

Thanks to the guys who integrated gitlab I was able to suceed with my RSS feed integration:

It´s still work in progress, but for now, it works quite well.

Thanks for your help, @it33!

Markus

11

@elpatron68 fantastic! Thanks for sharing the project with the community.

Would you be open to adding a README file, so we could share this on our integration site?

12

@it33 finally, we have a README now. I´d really love to see someone try it out before we share this. Please report any issues (or better: fork & fix them?)! :wink:

13

Thanks @elpatron68!

Looks good, just let us know when you’re comfortable sharing, would be a valuable addition to community resources,

1 Like
14

Hi @it33,

I have testet with several virtual machines against two different Mattermost servers; no issues.

BTW: I can´t push to Town Square via webhook. I assume it´s meant to be that way?

15

Thanks @elpatron68!

Webhooks should work with Town Square, we definitely know of other instances where they’re set up that way. Can you let us know on which step they’re not working?

If you’re system feels otherwise stable, it might be beneficial sharing early. Mattermost apps are getting issues filed and pull requests from the community for improvements, and there are probably people in the community who’d be happy to hear about what you’ve started,

16

nice :innocent: perfect