Showing posts with label sockets. Show all posts
Showing posts with label sockets. Show all posts

Thursday, February 23, 2012

Featureless Sockets

Okay, so the title is a bit of a stretch. Dart Sockets are far from featureless. As some of you may or may not know, I've decided to take a segue in my EchoServer towards working on a basic MUD and MUDLib written in Dart. It will be far from full featured, and far from fully functional but it will be the basis and a nice little start.

Within a few minutes of converting the EchoServer to be the basis of the MUD driver, I've already run into an issue of a missing feature with Sockets in Dart. I'm unable to get the host of an incoming connection. That is, I can't find out who is connecting to me. While it's a trivial matter for a MUD, or many other services, one would expect such functionality for various uses including HTTP server logging, etc.

After digging through the bleeding edge to see if it may have been recently included but just not yet updated in the API, I still can't find any reference to such properties. So as such, I've created a bug report over in the Dart issue tracker. If you're interested, or want to star the report you can view it here at: Issue 1819

As I progress with the Dart core and MUDLib I'm sure I'll find some additional bugs to report, or just missing features I can think could be added. Additionally I'll keep progress reports available on here and eventually release the source on GitHub or something similar.

Tuesday, February 21, 2012

Serving Sockets... The Salad

I haven't had the opportunity to do much with Dart lately, due to scheduling. However this afternoon I had a little chance to write some more. I decided that I wanted to modify the EchoServer to maintain the connection and close the connection only when receiving a specific command. I also wanted to add a command to shutdown the server itself completely. In order to properly accommodate this, I needed to refactor the server code a little. I wrapped the ServerSocket in a manager class:

#import('dart:io');

class ServerManager {
  ServerSocket _listenServer;
  
  ServerManager() {
    _listenServer = new ServerSocket("127.0.0.1", 5700, 0);
    
    _listenServer.connectionHandler = this._handleConn;
  }

  void _handleConn(Socket conn) {
    StringInputStream clientIn = new StringInputStream(conn.inputStream);
    
    clientIn.lineHandler = () {
      String input = clientIn.readLine();
      print("Received: $input");
      if(input.toLowerCase() == 'stop') {
        String cls = "** Stopping Server. Closing connection **\n";
        conn.writeList(cls.charCodes(), 0, cls.length);
        conn.close();
        _listenServer.close();
        print("** Stopping Server! **");
      } else if(input.toLowerCase() == 'exit') {
        String cls = "** Closing connection to client **\n";
        conn.writeList(cls.charCodes(), 0, cls.length);
        conn.close();
        print("** Closing connection to client. **");
      } else {
        String output = "${input.toUpperCase()}\n";
        conn.writeList(output.charCodes(), 0, output.length);
        print("Sent: $output");
      }
    };
    
  }
}

void main() {
  ServerManager sMan = new ServerManager();
 
}

So as you can see I made a few changes from my original EchoServer. As mentioned above I wrapped the server in a manager class, this enables me to easily close the server socket without using a global variable. In addition I added a couple clauses to check for the 'stop' or 'exit' commands which will stop the server or just close the client connection respectively. And finally I stopped pulling the output stream of the sockets directly, and instead use the writeList methods directly on the socket itself. I wasn't gaining any real benefit by creating an additional variable for the socket's OutputStream, so I just dropped it altogether.

Now as is, the above will run and accept connections and echo any new lines until the stop or exit commands are received. If the exit command is received, then the server will close the connection to that socket. If stop is received it will close the connection to that socket and then tell the server to close. However because of the event driven nature of the server, the Sockets and ServerSockets are not blocking. That is, even without adding any additional threads (Isolates), we can accept connections from multiple sources. If you open up multiple telnet connections to the host, you can see how you can send data and receive responses on each connection independent of the other.

But this also leaves us with a small issue. If we tell the server to stop from one telnet session while the other is still active.. the server will accept the stop command, and it will schedule the ServerSocket to be stopped, but not until the other socket has been closed. Try it out and you will see that the connection in which we issue the stop command is disconnected, and the console will indicate that the server is stopping. But the other telnet session will remain active until we issue an exit or stop command. Only once the 2nd session is closed will the server stop. And if for some reason the other session does not terminate properly (for instance connection drops or the telnet application is closed before issuing an exit/stop command) then the server will hang, not accepting new connections but not terminating either (assuming eventually the socket will time out but potentially not since I do not have those error handlers in place either).

This may be the desired situation with some servers to shut them down gracefully for instance, however in our EchoServer we want it to shut down immediately if it receives the stop command. So we'll need to keep a list of active connections and iterate through them and close each one, then stop the server. So I ended up with the following:

#import('dart:io');

class ServerManager {
  ServerSocket _listenServer;
  List _socketList;
  
  ServerManager() {
    _socketList = new List();
    _listenServer = new ServerSocket("127.0.0.1", 5700, 0);
    
    _listenServer.connectionHandler = this._handleConn;
  }
  
  void sendStops() {
    List cls = "** Server received stop request. Closing connection to client **\n".charCodes();
    
    while(!_socketList.isEmpty()) {
      Socket conn = _socketList.removeLast();
      conn.writeList(cls, 0, cls.length);
      conn.close();
    }
  }
  
  void _handleConn(Socket conn) {
    _socketList.add(conn);
    StringInputStream clientIn = new StringInputStream(conn.inputStream);
    
    clientIn.lineHandler = () {
      String input = clientIn.readLine();
      print("Received: $input");
      
      if(input.toLowerCase() == 'stop') {
        sendStops();
        _listenServer.close();
        print("** Stopping Server! **");
      } else if(input.toLowerCase() == 'exit') {
        String cls = "** Closing connection to client **\n";
        conn.writeList(cls.charCodes(), 0, cls.length);
        int sockInd = _socketList.indexOf(conn);
        _socketList.removeRange(sockInd, 1);
        conn.close();
        print("** Closing connection to client: $sockInd **");
      } else {
        String output = "${input.toUpperCase()}\n";
        conn.writeList(output.charCodes(), 0, output.length);
        print("Sent: $output");
      }
    };
    
  }
}

void main() {
  ServerManager sMan = new ServerManager();
 
}

As you can see I also added a method sendStops just to iterate through all the sockets, popping them out of list and sending them the stop notice and disconnecting them. I made this separate from the actual stopping of the server in case it should ever be required for any other reason as well. Initially I tried using a Set to hold just unique connections, and provide easier way of removing elements however I found out that there's an issue with Set's in that any values stored in a set must implement Hashable. This wasn't added in the API documentation and it was only after a little digging through the DartBug page and Newsgroup that I found this is 'expected' behaviour. As such, I had to use the list. For a specific 'exit' command I have to get the index of the value and remove it from the list with removeRange with a size of 1 element. I also setup the broadcast message directly to a List of Int's immediately just to avoid having to convert it multiple times as I iterate through the connections. While still missing any error handling, etc. I'm rather pleased with how the server is progressing and in some ways it conjures up images of the old school MUD's. Maybe a project to play with?

Thursday, February 16, 2012

Serving Sockets... The Soup Crackers

So as I mentioned in my previous post, there is more than one way to communicate through sockets. Below I have show 3 different, though similar ways to make use of sockets. This is far from exhaustive and please note that this is for demonstration purposes only and as is, should not be used in production code of any kind. They're missing error checking, stream verification, etc.

I am interested if any one has any comments or feedback as to use cases where they would choose to use one of the particular methods over another. Everything below is fairly procedural and could probably be implemented much cleaner as Objects, but as mentioned this is purely a test case using the EchoServer I wrote a couple posts back.

#import("dart:io");

void main() {
  sockets_with_handlers();
  
  sockets_with_direct_streams();

  sockets_with_socket_streams();
}

void sockets_with_handlers() {
  Socket usingHandlers = new Socket("127.0.0.1", 5700);
  String test = "SocketHandlers Test String\n";
  
  // Called when we successfully connect
  usingHandlers.connectHandler = () {
    print("Handers: Connected");
  };
  
  // Called when we can write to the socket.
  usingHandlers.writeHandler = () {
    usingHandlers.writeList(test.charCodes(), 0, test.length);
    print("Handers Sent: $test");
  };
  
  // Called when we receive from the socket.
  usingHandlers.dataHandler = () {
    int availBytes = usingHandlers.available();
    List buff = new List(availBytes);
    usingHandlers.readList(buff, 0, availBytes);
    print("Handers Received: ${new String.fromCharCodes(buff)}");
  };
  
  // Called when the input stream from socket is closed.
  usingHandlers.closeHandler = () {
    print("Handers: End of stream. Closing connection");
    usingHandlers.close();
  };
}

void sockets_with_direct_streams() {
  Socket usingStreams = new Socket("127.0.0.1", 5700);
  String test = "Direct Socket Streams test string\n";
  
  // To be 'pure' without the handlers we assume connection is successful.
  // NOTE: Not recommended!
  print("Direct Streams: Connected");
  // Wrap the input stream in StringInputStream so we can
  // make use of the convenience functions.
  StringInputStream inStream = new StringInputStream(usingStreams.inputStream);
  OutputStream outStream = usingStreams.outputStream;
  
  // Write to our stream
  outStream.write(test.charCodes());
  print("Direct Streams Sent: $test");
  
  // Use our handle wrapper to read lines
  // Saves us from using dataHandler directly
  inStream.lineHandler = () {
    String input = inStream.readLine();
    print("Direct Streams Received: $input");
  };
  
  // All bytes have been read and input stream is closed.
  inStream.closeHandler = () {
    print("Direct Streams: End of stream. Closing connection");
    usingStreams.close();
  };
}

void sockets_with_socket_streams() {
  Socket usingSocketStreams = new Socket("127.0.0.1", 5700);
  String test = "Socket In/Out Streams test string\n";
  
  // To be 'pure' without the handlers we assume connection is successful.
  // NOTE: Not recommended!
  print("Socket Streams: Connected");
  
  SocketInputStream inStream = new SocketInputStream(usingSocketStreams);
  SocketOutputStream outStream = new SocketOutputStream(usingSocketStreams);
  
  // Write to our stream.
  outStream.write(test.charCodes());
  print("Socket Streams Sent: $test");
  
  // Called when new data arrives in our SocketInputStream
  inStream.dataHandler = () {
    List buff = inStream.read();
    print("Socket Streams received: ${new String.fromCharCodes(buff)}");
  };
  
  // Input stream has been closed. Make sure output stream
  // and socket itself are also closed.
  inStream.closeHandler = () {
    print("Socket Streams: End of stream. Closing connection");
    outStream.close();
    usingSocketStreams.close();
  };
}
Direct Streams: Connected
Direct Streams Sent: Direct Socket Streams test string

Socket Streams: Connected
Socket Streams Sent: Socket In/Out Streams test string

Handers: Connected
Handers Sent: SocketHandlers Test String

Direct Streams Received: DIRECT SOCKET STREAMS TEST STRING
Socket Streams received: SOCKET IN/OUT STREAMS TEST STRING

Handers Received: SOCKETHANDLERS TEST STRING

Direct Streams: End of stream. Closing connection
Socket Streams: End of stream. Closing connection
Handers: End of stream. Closing connection

One of the first things you'll notice with the output is that due to the non-blocking nature of the callbacks, some of the calls happened in near-parallel. In the sockets_with_socket_streams function, while I didn't specifically make use of it, we could have also wrapped the SocketInputStream with a StringInputStream as well, similar to what we did in the sockets_with_direct_streams function.

Serving Sockets... The Soup

Alright, first thing first. Announced during yesterday's Episode 2 of Dartisans Hangout, Dartium binaries are being relased. Initially for Mac OS X and Linux with Windows binaries to follow soon. I won't go on about this much as it has already been mentioned again and again, and again.

One thing to note however, is that currently the Linux Build is a 32-bit version so if you're running a 64-bit OS you will need to ensure you have the proper 32-bit libraries downloaded and installed. Or alternatively, wait for the 64-Bit builds to arrive.

Continuing with my small series on Sockets in the Dart:IO library, I thought well I have an EchoServer written, albeit extremely primitive, my next 'logical' step is to make a client which connects to the server, sends a string, receives the echo response and disconnects. This should be pretty straight forward, since it was so easy to make the server... Oh boy was I ever wrong about that. Not that Sockets are overly complex or anything, they just have many more ways of accomplishing the same thing, but they can't be used in conjunction as I found out with the following:

#import("dart:io");

void main() {
  // Create a new socket connecting to localhost and port 5700
  // the same port as our echo server we wrote is running on.
  Socket conn = new Socket("127.0.0.1", 5700);
  StringInputStream inputStr;
  OutputStream outputStream;
  
  // method is called when connection is established.
  conn.connectHandler = () {
    print("Now Connected");
    inputStr = new StringInputStream(conn.inputStream);
    outputStream = conn.outputStream;
    
    String test = "This is a simple test\n";
    outputStream.write(test.charCodes());
    print("Sending: $test");
    
    // We wrapped the input stream for this easier reading
    // using the lineHandler as opposed to bulk data.
    inputStr.lineHandler = () {
      String input = inputStr.readLine();
      print("Recieved: $input\n");
    };
  };
  
  // Called when the last byte of data has been read from the socket.
  // Socket potentially still open for writing.
  conn.closeHandler = () {
    print("Connection closed. Last byte of data has been read from stream");
    conn.close();
  };
  
}
Now Connected
Unhandled exception:
StreamException: Cannot get input stream when socket handlers are used
 0. Function: '_Socket@14117cc4.get:inputStream' url: 'dart:io' line:4231 col:9
 1. Function: '::function' url: 'src/dart/echo/EchoClient.dart' line:13 col:54
 2. Function: '_Socket@14117cc4.firstWriteHandler' url: 'dart:io' line:4274 col:64
 3. Function: '_SocketBase@14117cc4._multiplex@14117cc4' url: 'dart:io' line:3898 col:23
 4. Function: '_SocketBase@14117cc4.function' url: 'dart:io' line:3996 col:59
 5. Function: 'ReceivePortImpl._handleMessage@924b4b8' url: 'bootstrap_impl' line:1734 col:22

So as we can see, this didn't work quite as cleanly as we'd have liked. Apparently the issue above is with my mixing the Socket handlers, such as connectHandler and closeHandler, while also pulling the socket's input and output streams themselves. In the case of the InputStream wrapping it in a StringInputStream handler to make dealing with input a little easier since we only expect text.

Initially I thought this error was telling me that I cannot use any socket handlers, if I wanted to pull the IO streams directly from the Socket. However I only just now stumbled upon an easier solution. The error isn't with the use of connectHandler and closeHandler on the socket. Apparently only the closeHandler is triggering the error. So I can move that method to be called on the StringInputStream instead. So a quick re-write gives me the following:

#import("dart:io");

void main() {
  // Create a new socket connecting to localhost and port 5700
  // the same port as our echo server we wrote is running on.
  Socket conn = new Socket("127.0.0.1", 5700);
  StringInputStream inputStr;
  OutputStream outputStream;
  
  // method is called when connection is established.
  conn.connectHandler = () {
    print("Now Connected");
    inputStr = new StringInputStream(conn.inputStream);
    outputStream = conn.outputStream;
    
    String test = "This is a simple test\n";
    outputStream.write(test.charCodes());
    print("Sending: $test");
    
    // We wrapped the input stream for this easier reading
    // using the lineHandler as opposed to bulk data.
    inputStr.lineHandler = () {
      String input = inputStr.readLine();
      print("Recieved: $input\n");
    };

    // Called when the last byte of data has been read from the socket.
    // Socket potentially still open for writing.
    inputStr.closeHandler = () {
      print("Connection closed. Last byte of data has been read from stream");
      conn.close();
    };
  };
  
}
Now Connected
Sending: This is a simple test

Recieved: THIS IS A SIMPLE TEST

Connection closed. Last byte of data has been read from stream

Well then, that's working so much better. And in fact I could leave it here.. and will for the time being. However later, perhaps tomorrow, I will write a continuation of this post on some of the other methods of using and communicating with sockets. They are plentiful.

Wednesday, February 15, 2012

Servering Sockets... The Appetizer

So I decided it was time to start playing around with some Sockets. As we already know, I'm not a huge fan of the chat sample that's available, as I find it a little too much. Fortunately the Sockets and ServerSockets are not too different from the Process API, so keeping that in mind I wrote a dead simple echo server. This does not adhere to any protocols or anything of that nature. It just accepts input from a client and sends it back to the client. Truth be told, I should be a little ashamed of the following code as its far from complete, far from safe, and far from extendable. But it was a good starting point and I'll be able to now take it and re-factor it convert it into something usable. The goal was simple, to write a simple server and have evidence that communication was bi-directional.

So first I'll provide the code and then a little discussion about it.

#import('dart:io');

void main() {
  // Create a server and bind it to an address and port number.
  ServerSocket listenServer = new ServerSocket("127.0.0.1", 5700, 0);
  
  // This is called when a connection is received.
  listenServer.connectionHandler = handleConn;
}

// Socket conn is the incoming connection.
void handleConn(Socket conn) {
  // Wrap the input from the connection around a StringInputStream to make reading it easier.
  StringInputStream clientIn = new StringInputStream(conn.inputStream);
  // Get the output stream too so we can write back to the client.
  OutputStream clientOut = conn.outputStream;
  
  // Line handler is called when a new line is received to the StringInputStream
  // which we've wrapped around the InputStream from the socket.
  clientIn.lineHandler = () {
    String input = clientIn.readLine();
    print("Received: $input");
    String output = "${input.toUpperCase()}\n"; 
    clientOut.write(output.charCodes());
    print("Sent: $output");
    conn.close();
  };
  
}

So as you can see above the code is very short and simple. I recommend running this in the IDE as currently the process will run forever. Later on I may add a command to shutdown the server from the client. To test out the above code start the server with the DartVM then open a telnet session. Telnet to 'localhost:5700' Then simply type a line of text and hit enter. You should get the same line of text sent back to you but uppercase.

Basically we create the server, and then just handle any incoming connections with it. The connectionHandler is called when a new connection is established to the server, and the function is passed the Socket to the incoming connection.
From that socket we get the InputStream and OutputStreams. For the InputStream we wrap it in a StringInputStream which simplifies reading strings from the connection, and since we're not really worried about receiving any bulk or binary data as such.

Next, thanks to the StringInputStream, we setup a lineHandler which is called when a new line of text is received by the stream. Then we use readLine to store that line in a variable.

We print what we've received to the console, then create a new string which is uppercase version of the line we just received, and add a new line character to the end of it. We want to add the newLine character because when we use the readLine method, it strips the new line character for us automatically. In this case I wanted to return an uppercase string of the original just for visual confirmation that what is being returned is different in some way from the original so it can't be blamed as a local echo by the telnet client or anything like that.

Next we write the output back to the output stream. Because the OutputStream is not wrapped by a String stream we need to convert our string into a list of character codes which are then in turn sent back through the SocketOutputStream. We also write to the console what we sent.

Finally we close the socket. In a later example we may leave this open until we send a specific comment to terminate the client connection, or specifically have the client close the connection instead.

As I mentioned, this example is pretty much as simple as you can get. It also is missing any error handling, any protocol conformation, but it gave me a comfortable start/introduction to sockets in general. If you have any suggestions or comments regarding the code by all means I eagerly look for feedback or suggestions.