Search code examples
pythonpysparkspark-streamingdatabrickstweepy

Connection with client not establishing with socket.accept()


I'm doing a project for class where I stream data from Twitter using Databricks and when it reaches s.accept() it seems to get stuck there, running indefinitely:

Code:

def sendTweets(c_socket):

 auth = OAuthHandler(API_key, API_secret_key)
 auth.set_access_token(access_token, access_token_secret)
 stream = Stream(auth, getTweets(c_socket))
 stream.filter(track=['covid','covid-19','covid19','coronavirus']) 

s = socket.socket()
host = "127.0.0.1"
port = 3333
s.bind((host, port))
s.listen()
c, addr = s.accept()
sendTweets(c)

Sorry if it's a very dumb question


Solution

  • This answer is untested and probably won't work but hopefully it'll give you an idea.

    You could try the following (not my best code):

    import threading
    def sendTweets(c_socket):
    
     auth = OAuthHandler(API_key, API_secret_key)
     auth.set_access_token(access_token, access_token_secret)
     stream = Stream(auth, getTweets(c_socket))
     stream.filter(track=['covid','covid-19','covid19','coronavirus']) 
    
    def StartSocket():
        global c, addr
        c, addr = s.accept()
    
    s = socket.socket()
    host = "127.0.0.1"
    port = 3333
    s.bind((host, port))
    s.listen()
    thread = threading.Thread(target=StartSocket) #fork off the server code
    thread.start()
    sendTweets(c)