pashov.net/code|Blog Archives

Tag Archives: c#

Others

Switching from C# to Ruby

Published by:

In March 2016 i switched jobs leaving Progress and joining Receipt Bank. So i had to switch the whole tech stack. In Progress my hands were mainly busy with C#, Web API, Visual Studio, MS SQL, MSMQ and the rest of the MS technologies. In Receipt Bank i we use Ruby (mainly) with Rails as a Web framework. I never actually wanted to switch but it happened this way – i liked the company and had to.

Changing the language is easy. Changing the stack and the tools is not.

When you are a more seasoned programmer, you should be able to switch pain-free. Well, it depends. If you switch from C# to Java or from Ruby to Python it should be fairly easy. It is not that easy if you switch from C# to Ruby as it is not only the language that you will have to learn. You will probably have to drop Windows and adopt Linux or OSX. You will probably have to switch from IDE as VS to a text editor as Sublime, Vim etc. You will most probably use PostgreSQL or MySQL instead of MSSQL. The database change is not that dramatic as the later two being SQL, they share a lot in common with MSSQL. However, dropping the beloved IDE in favor for a text editor is a pain, at least in the beginning. You quickly realise that you may be as productive with a text editor as with an IDE. A power Vim user is at least as productive as a power VS user. The OS change is kind of a dramatic also. I started using OSX and felt great pain in the beginning. Most of my time was in the console which is quite contrary to the Windows experience. However, when you get used to the almighty console, you never look back.

How is the language different?

#1 Ruby is interpreted and C# is compiled. So in Ruby everything is just working until..you hit a problem. C# compiler will complain during compile time if your code is no good.
#2 C# has static typing and Ruby does not. This is not one sided though. Matz (Ruby creator) said that some typing is coming in Ruby 3 and C# introduced var and dynamic to provide for more flexibility. However, the general picture is that Ruby is interested in behaviour and not static properties. This gives great flexibility and i find this rather liberating. To avoid shoot-in-the-leg situations, you have to have very high test coverage with flexible types.
#3 Testing. Flexible languages as Ruby provide for much better testing than C#. In C# you often have one interface with one implementation class which i find stupid. However, you have to do that so you can mock that class in tests. If Ruby you mock pretty everything easily and don’t really need interfaces (there aren’t any also:) ).
#4 Functional programming has much deeper roots in Ruby. C# has made big steps also with LINQ but Ruby is superior in this domain, especially when working with collections.
#5 Ruby has a very powerful feature called blocks. These are like callbacks to functions and are really powerful. There is hardly a class that you can’t find blocks.
#6 Syntax. Ruby is much more minimalistic than C#. Syntax is weird in the beginning but you soon start to love it. Little parens are required and there is no return statement. You can single line a lot of stuff to make code more concise. Syntax is important to write readable code and Ruby definitely has the syntax for that.

What else

I would say that the ruby community is much more vibrant than the C# one. Ruby was created with developer’s happiness as a main goal and developers love it. Ruby is religion for some. Matz is accepted as a rock star in the community. There is nothing like that with the MS technologies.  Giving back to the community is a law for many. That’s why you will find many Ruby developers spend time coaching and teaching others for free. They also contribute to open source big time.

True Ruby developers are committed to quality very much. They also keep code clean and neat with great care. Code must not only work but be beautiful. Fellow developers are very fast to growl if you put one extra coma somewhere. Most of the C# developers i have seen are a bit messy. Quality is also appreciated but readability and code beauty not that much. As i said happiness is a founding principle for the language and you can tell that.

C# is preferred by corporations and Ruby by start ups. So choosing the language will have impact on the company that you will be working in.

Conclusion

I find switching to Ruby a huge win. Despite the pain in the beginning, the language is very elegant and powerful. There are enough good tools also out there to get the job done. The biggest advantage however is the community. Being part of that is guaranteed to higher your level.

 

.NET Coding

Download large files with Web Api as a relay

Published by:

Consider the following scenario: you have a file storage in the cloud (google docs, openstack object storage etc.) and you need to provide your app consumers a way to download the files through your app. The challenges here are mainly memory related – if you get files in memory and then return them in the response, your app will soon be crushed with OutOfmemory Exception. So you need a way to turn your app into a relay which is kind of a pipe through which bytes flow from the file storage to the client without buffering any of the content. I guess you are already thinking streams and you are right.

Change the buffer policy

Web api by default will try to buffer the content that you are trying to download and that needs to be changed. You will need to be inherit from IHostBufferPolicySelector  or inherit from the only implementor of this interface (WebHostBufferPolicySelector) and override its methods :

nbsp;

public class WebHostBufferPolicySelector : IHostBufferPolicySelector
{
    public virtual bool UseBufferedInputStream(object hostContext);
    public virtual bool UseBufferedOutputStream(HttpResponseMessage response);
}

You will also need to override one of the two methods, depending if you want unbuffered content for download or upload. In our case we need UseBufferedOutputStream. A possible implementation would be:

public class NoBufferPolicy : WebHostBufferPolicySelector
    {
        public override bool UseBufferedOutputStream(HttpResponseMessage response)
        {
            if (response.RequestMessage.RequestUri.LocalPath.Contains("download"))
            {
                return false;
            }

            return base.UseBufferedOutputStream(response);
        }
    }

So now we use not buffered content when anybody hits a route that contains download and buffered for any other case. After we have done that we need to register our class in GlobalConfiguration:

 GlobalConfiguration.Configuration.Services.Replace(typeof(IHostBufferPolicySelector), new NoBufferPolicy());

Back to the controller

In the usual case the code in your controller should be something like:

 

var client = new HttpClient();
var responseWithHeadersOnly = await client.GetAsync(requestUrl, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false);

Two things are worth mentioning here – the second parameter of the GetAsync method – HttpCompletionOption.ResponseHeadersRead which basically reads only the headers of the response and not the content. The second thing is the ConfigureAwait(false) which has to do with how async and await play in a web app.

Here is the time to check if the response is ok. Once we have the response, we need to get hold of the content stream without putting it in memory:

Stream streamToReadFrom = await responseWithHeadersOnly.Content.ReadAsStreamAsync();

After we get the headers, make sure that everything is ok (no service unavailable, server errors etc.) and have control over the stream it is time to start streaming to the client. We will use a push approach in this case, contrary to the traditional pull. Luckily web api gives us the tools for that – we will use the PushStreamContent object for our response content. What it does is that it reads from a stream and pushes the bytes to the client chunk-wise. It has a couple of overloads but it basically uses a function that takes a Stream, HttpContext and TransportContext as params and may return void or a Task. Let’s see some code:

 var client = new HttpClient();
var streamToReadFrom = await client.GetAsync(requestUrl, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false);

                var buffer = new byte[65536];
                var result = Request.CreateResponse();
                int bytesRead = 0;

                result.Content = new PushStreamContent((stream, content, context) =>
                {
                    while ((bytesRead = streamToReadFrom.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        stream.Write(buffer, 0, bytesRead);
                    }

                    stream.Close();
                    streamToReadFrom.Close();
                    client.Displose();
                });

                result.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
                result.Content.Headers.ContentDisposition.FileName = filename;
                result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
                return result;

Every time you go through the while loop, you read bytes from the incoming stream and pass them to the outgoing stream.

The flow here is as follows – a request comes in (1), you read the headers of the requested file from the file storage provider (2), return response from the method (return result; line (3)), file starts to download (stream.Write(buffer, 0, bytesRead); (4)), file is done downloading (stream.Close(); (5)). The whole thing ends when you close the outgoing stream and not when you return from the method.

That should be all to turn your web api in a working and scalable file download relay. You can find the source code HERE under the LargeFilesWebApi folder

.NET Coding

Construct Dynamic Filters with C#

Published by:

Filtering data with C# is pretty straight-forward when using LINQ – you get results with one-liners as

var result=cars.Where(a=>a.carMake=="Fiat" && a.speed>100);

However, we know the filtering properties (carMake and speed) and the filtering conditions (equals with carMake and greater than with speed) in advance. The more interesting case, however, is when we don’t know the filtering properties, their number and the filtering criteria in advance but have to construct them in real time. Consider the case when you have data objects with many properties and allow the user on the front end to filter it according to his/her needs. In this case Dynamic Expression Construction comes to rescue.
LINQ extension methods work with predicates (or anonymous functions). Fortunately, C# provides very powerful api to build Expression trees, which can be compiled to predicates in runtime and passed to the LINQ extension methods afterwards.
Let’s see an example. Suppose that we have a list of cars and we want to filter them dynamically for whatever purpose – here is our Car object

 
public class Car
    {
        public string Model { get; set; }
        public string Make { get; set; }
        public string CountryOfOrigin { get; set; }
        public string CountryOfProduction { get; set; }

        public int MaxMilesPerHour { get; set; }
        public int NumberOfSeats { get; set; }
        public int MilesPerGallon { get; set; }
        public double WeightInKg { get; set; }

        public decimal BasePrice { get; set; }
        public decimal FullPrice { get; set; }
    }

With 10 properties, we can filter a collection of cars in many possible ways.
Next we need a class that would represent a filter – so we need a property name for comparison, value to be compared to and operation (equal, less than etc). Let’s call it ExpressionFilter

public class ExpressionFilter
    {
        public string PropertyName { get; set; }
        public object Value { get; set; }
        public Comparison Comparison{ get; set; }
    }

Comparison is nothing more than a enum with possible operations

public enum Comparison
    {
        Equal,
        LessThan,
        LessThanOrEqual,
        GreaterThan,
        GreaterThanOrEqual,
        NotEqual,
        Contains, //for strings
        StartsWith, //for strings
        EndsWith //for strings
    }

Next comes the real thing – we need a class which can take a list of ExpressionFilters, do its thing and return an Expression tree composed of these filters, which we can use as a parameter in any LINQ extension method after compiling it. Let’s have a class that constructs And expression trees. We will call it ConstructAndExpressionTree

 
public static Expression<Func<T, bool>> ConstructAndExpressionTree<T&gt(List<ExpressionFilter> filters)
        {
            if (filters.Count == 0)
                return null;

            ParameterExpression param = Expression.Parameter(typeof(T), "t");
            Expression exp = null;

            if (filters.Count == 1)
            {
                exp = ExpressionRetriever.GetExpression<T>(param, filters[0]);
            }
            else
            {
                exp = ExpressionRetriever.GetExpression<T>(param, filters[0]);
                for (int i = 1; i < filters.Count; i++)
                {
                    exp = Expression.And(exp, ExpressionRetriever.GetExpression<T>(param, filters[i]));
                }
            }

            return Expression.Lambda<Func<T, bool>>(exp, param);
        }

The class is generic so it can be used with any type. The code is pretty straight-forward -> we start with the ParameterExpression which is the t=> part of the predicate. Then we iterate through the filters and start building the actual expression called exp here. You probably notice that we use a helper method ExpressionRetriever.GetExpression to construct the separate expressions (like a.speed>100), which we stick with Or binary operator aferwards. Here is the code of ExpressionRetriever.GetExpression:

public static class ExpressionRetriever
    {
        private static MethodInfo containsMethod = typeof(string).GetMethod("Contains");
        private static MethodInfo startsWithMethod = typeof(string).GetMethod("StartsWith", new Type[] { typeof(string) });
        private static MethodInfo endsWithMethod = typeof(string).GetMethod("EndsWith", new Type[] { typeof(string) });

        public static Expression GetExpression<T>(ParameterExpression param, ExpressionFilter filter)
        {
            MemberExpression member = Expression.Property(param, filter.PropertyName);
            ConstantExpression constant = Expression.Constant(filter.Value);
            switch (filter.Comparison)
            {
                case Comparison.Equal:
                    return Expression.Equal(member, constant);
                case Comparison.GreaterThan:
                    return Expression.GreaterThan(member, constant);
                case Comparison.GreaterThanOrEqual:
                    return Expression.GreaterThanOrEqual(member, constant);
                case Comparison.LessThan:
                    return Expression.LessThan(member, constant);
                case Comparison.LessThanOrEqual:
                    return Expression.LessThanOrEqual(member, constant);
                case Comparison.NotEqual:
                    return Expression.NotEqual(member, constant);
                case Comparison.Contains:
                    return Expression.Call(member, containsMethod, constant);
                case Comparison.StartsWith:
                    return Expression.Call(member, startsWithMethod, constant);
                case Comparison.EndsWith:
                    return Expression.Call(member, endsWithMethod, constant);
                default:
                    return null;
            }
        }
    }

Expression class provides methods for the standard operations as equal, not equal, less than etc. but notice also the last three cases where we use Expression.Call to call the string methods Contains, StartsWith and EndsWith. Here lies a big potential for extension as you can use whatever methods you like defined on any type. You only should get their MethodInfo through reflection.
So sticking it all together (i skip the part where i populate the cars collection):

      static void Main(string[] args)
        {
            var filters = new List<ExpressionFilter> 
            { 
                new ExpressionFilter
                {
                    PropertyName="CountryOfOrigin",
                    Comparison=Comparison.StartsWith,
                    Value="Fr"
                },
                new ExpressionFilter
                {
                    PropertyName="MaxMilesPerHour",
                    Comparison=Comparison.GreaterThanOrEqual,
                    Value=190
                }
            };

            var expressionTree= ExpressionBuilderHelper.ExpressionBuilder.ConstructAndExpressionTree<Car>(filters);
            var anonymousFunc = expressionTree.Compile();
            var result = cars.Where(anonymousFunc);
        }

You can see that the result is a car that has MaxMilesPerHour>190 and CountryOfOrigin that starts with ‘Fr’
result

.NET

StringBuilder vs. Plus Operator

Published by:

One of the first things every developer learns is handling strings. String operations are so common that there is hardly a day without having to handle strings, so knowing the nitty-gritty of string handling is vital for every good developer.

It is something like a dogma to use StringBuilder when handling strings and not the + operator or String.Concat (which is + under the hood) because StringBuilder is way faster. Strings are immutable in C# so every time you want to change a string, you basically create a new string. StringBuilder solves this problem by keeping an array where it keeps the string chars and auto grows the array when needed (pretty much same principle as how Lists work in C#). The string object is then derived when calling the ToString method.

Although better performance is generally the case with StringBuilder VS the plus  operator, with string concatenation of only a few strings, i find the + operator  much more readable and the performance drain is close to zero so nothing to worry about. Consider the following example:


Stopwatch sbWatch = new Stopwatch();
Stopwatch plusOperatorWatch = new Stopwatch();
var destinationStringBuilder = new StringBuilder();
string destinationString = String.Empty;
string sourceString = new String('X', 20);

sbWatch.Start();
for (int i = 0; i <= 10000; i++)
{
    // Concat strings 3 times using StringBuilder
    destinationStringBuilder.Append(sourceString);
    destinationStringBuilder.Append(sourceString);
    destinationStringBuilder.Append(sourceString);
    destinationStringBuilder.Clear();
}
sbWatch.Stop();

plusOperatorWatch.Start();
for (int i = 0; i <= 10000; i++)
{
    // Concat strings 3 times using + operator concatenation
    destinationString = sourceString;
    destinationString += sourceString;
    destinationString += sourceString;
}
plusOperatorWatch.Stop();

On my machine the clock for StringBuilder shows 0 milliseconds and the clock for the plus operator shows 3 milliseconds which is very small difference for doing three string concatenations 10 000 times and in my opinion the + operator is much more readable.

The case, however, for StringBuilder is when we build larger texts out of smaller strings. If we remove

destinationStringBuilder.Clear();

from the first loop and convert

destinationString = sourceString;

to

destinationString += sourceString;

in the second loop

performance becomes an issue with StringBuilder outperforming + operator in the range of hundreds of times. StringBuilder takes 3 milliseconds and the + operator takes a little under 17 seconds.

So rule of thumb is if you have to do string manipulations with up to 5 strings go for the + operator for better readability, otherwise use StringBuilder for performance.

Consider the following example - a common practice is to write logs somewhere so i find much more readable to do something like :

var message="User " + user.userName + " tried to access resource " + resource.name + " at " + DateTime.Now;
Logger.info(message);

instead of:

var sb=new StringBuilder();
sb.Append("User ");
sb.Append(user.userName);
sb.Append(" tried to access resource ");
sb.Append(resource.name);
sb.Append(" at ");
sb.Append(DateTime.Now);
Logger.info(sb.ToString());

And performance difference in this case will be close to zero.