4GuysFromRolla.com :: An Extensive Examination of LINQ Article Series

An Extensive Examination of LINQ: Extending LINQ - Adding Query Operators

mitchell@4guysfromrolla.com (Scott Mitchell) — Wed, 7 Jul 2010 00:00:00 GMT

As discussed in earlier installments of this article series - most notably in An Introduction to LINQ and The Standard Query Operators - one of LINQ's primary components is its set of standard query operators. A query operator is a method that operates on a sequence of data and performs some task based on that data, are implemented as extension methods on types that implement the IEnumerable interface. Some of the standard query operators that we've explored throughout the articles in this series include: Count, Average, First, Skip, Take, Where, and OrderBy, among others.

While these standard query operators provide a great detail of functionality, there may be situations where they fall short. The good news is that it's quite easy to create your own query operators. Underneath the covers query operators are just methods that extend types that implement IEnumerable and iterate over the sequence performing some task, such as computing the total number of items in the sequence, computing the average, filtering the results, or ordering them. This article examines how to extend LINQ's functionality by creating your own extension methods. Read on to learn more!

An Extensive Examination of LINQ: Querying and Searching XML Documents Using LINQ to XML

mitchell@4guysfromrolla.com (Scott Mitchell) — Wed, 23 Jun 2010 00:00:00 GMT

XML is an increasingly popular way to encode documents, data, and electronic messages. Over the years Microsoft has offered a variety of libraries to facilitate creating, modifying, querying, and searching XML documents. LINQ to XML is a relatively new set of XML-related classes in the .NET Framework (found in the System.Xml.Linq namespace), which enable developers to work with XML documents using LINQ's features, syntax, and semantics. As discussed in an earlier article, Introducing LINQ to XML, LINQ to XML is a simpler and easier to use API than previous libraries. Because LINQ to XML can utilize LINQ's query syntax and assortment of standard query operators, LINQ to XML code is usually very terse and readable.

This article continues our look at LINQ to XML. Specifically, we explore how to query XML documents using axis methods as well as how to search and filter XML documents using both LINQ's Where method and XPath expressions. Read on to learn more!

Note: If you have not yet read Introducing LINQ to XML please do so before reading this article...

An Extensive Examination of LINQ: Introducing LINQ to XML

mitchell@4guysfromrolla.com (Scott Mitchell) — Wed, 25 Nov 2009 00:00:00 GMT

XML is an increasingly popular way to encode documents, data, and electronic messages. There are a number of ways to programmatically create, modify, and search XML files. Since its inception, the .NET Framework's System.Xml namespace has included classes for programmatically working with XML documents. For instance, the XmlReader and XmlWriter classes offer developers a means to read from or write to XML files in a fast, forward-only manner, while the XmlDocument class allows developers to work with an XML document as an in-memory tree representation.

LINQ to XML is a new set of XML-related classes in the .NET Framework (found in the System.Xml.Linq namespace), which enable developers to work with XML documents using LINQ's features, syntax, and semantics. Compared to .NET's existing XML APIs, LINQ to XML is a simpler, easier to use API. For a given task, LINQ to XML code is typically shorter and more readable than code that uses the XmlDocument or XmlReader/XmlWriter classes. And perhaps most importantly, LINQ to XML allows you to leverage your existing knowledge and familiarity with LINQ's standard query operators and query syntax.

This article is the first in a series of articles that examines LINQ to XML. This installment introduces the LINQ to XML API, examines some of the more pertinent classes in the System.Xml.Linq namespace, and shows how to perform a number of common XML tasks using the API. Read on to learn more!

An Extensive Examination of LINQ: Grouping and Joining Data

mitchell@4guysfromrolla.com (Scott Mitchell) — Wed, 5 Aug 2009 00:00:00 GMT

As discussed in An Introduction to LINQ, one of the cornerstones of LINQ is the standard query operators, which are a set of extension methods on the IEnumerable interface added to the .NET Framework version 3.5. The standard query operators can be applied to any enumeration - any collection of "things." In The Standard Query Operators installment we looked at some of the more common query operators, such as Where, Select, OrderBy, and others. Each standard query operator can be classified as a certain type of operator. There are aggregation operators like Count, Sum, and Max; element operators like First, Last, and ElementAt let you pick out a specific element from a sequence; and the ordering operators OrderBy and OrderByDescending order the elements of a sequence based on a specified sorting criteria.

Another class of query operators that we've yet to explore are grouping and joining operators. The grouping and joining operators work with two (or more) sequences and combine them together, much like how a JOIN in SQL combines records from two (or more) tables into a single resultset. Through LINQ's standard query operators (or via its query syntax), it is possible to perform: nested (or grouped) queries; cross joins, or the Cartesian product of two sequences; inner joins; and left outer joins.

This article explores the grouping and joining operators available in LINQ with a number of examples in both Visual Basic and C#. As with the previous installments in this article series, the complete code is available for download. Read on to learn more!

An Extensive Examination of LINQ: Using the Query Syntax

mitchell@4guysfromrolla.com (Scott Mitchell) — Wed, 15 Apr 2009 00:00:00 GMT

LINQ's standard query operators provide dozens of built-in techniques for programmatically enumerating, aggregating, selecting, and filtering data. These standard query operators are implemented as extension functions on the IEnumerable interface and therefore available to any object that implements this interface, which includes arrays, the collection classes in the System.Collections namespace, and objects that are composed of a set of enumerable items (such as a DataTable). The examples we've explored in this tutorial series thus far have primarily used these standard query operators in their extension method form, namely as methods applied to IEnumerable collections. For example, the following snippet uses this style of syntax to query a collection of Employee objects, returning a list of the employees' Name and Salary for those employees that make more than $100,000 per year, sorted by salary in descending order.


// C# - get the Name and Salary of employees that make more than $100,000, ordered from the highest paid on down

List emps = ...;

var SalaryReport = emps.Where( p => p.Salary > 100000M )

                       .OrderByDescending( p => p.Salary )

                       .Select( p => new { p.Name, p.Salary } );





' VB - get the Name and Salary of employees that make more than $100,000, ordered from the highest paid on down

Dim emps As List(Of Employee) = ...

Dim SalaryReport = emps.Where(Function(p) p.Salary > 100000) _

                       .OrderByDescending(Function(p) p.Salary) _

                       .Select(Function(p) New With {p.Name, p.Salary})

While the above syntax certainly works, it is not the most human-friendly syntax. For starters, it's quite verbose. It's also unlike the data querying syntax most developers are familiar with - SQL. To address these shortcomings, Microsoft added a number of language enhancements to C# 3.0 and Visual 9, which are the versions that shipped with the .NET Framework 3.5 and Visual Studio 2008. In addition to extension methods, implicitly typed variables, object initializers, lambda expressions, and anonymous types, Microsoft also added a new query syntax that allows developers to write LINQ queries in a SQL-like syntax. The following code is semantically equivalent to the example above, but uses the query syntax (or query expressions) instead of the extension method syntax:


// C# - get the Name and Salary of employees that make more than $100,000, ordered from the highest paid on down

List emps = ...;

var SalaryReport = from p in emps

                   where p.Salary > 100000M 

                   orderby p.Salary descending

                   select new { p.Name, p.Salary } );





' VB - get the Name and Salary of employees that make more than $100,000, ordered from the highest paid on down

Dim emps As List(Of Employee) = ...

Dim SalaryReport = From p In emps _

                   Where p.Salary > 100000 _

                   Order By p.Salary _

                   Select New With {p.Name, p.Salary}

This query syntax is another syntactic sugar language enhancement. When compiled, the above code is translated into calls to the corresponding standard query operators.

This article explores both C# and Visual Basic's query syntax. We'll discuss the mapping between the query syntax and LINQ's standard query operators, as well as see a number of query syntax examples. Read on to learn more!

An Extensive Examination of LINQ: The Standard Query Operators

mitchell@4guysfromrolla.com (Scott Mitchell) — Wed, 01 Apr 2009 00:00:00 GMT

Query operators are methods that work with a sequence of data and perform some task based on the data. They are created as extension methods on the IEnumerable interface, which is the interface implemented by classes that hold enumerable data. For example, arrays and the classes in the System.Collections and System.Collections.Generic namespaces all implement IEnumerable. In The Ins and Outs of Query Operators we looked at how to create your own query operator that, once created, can be applied to any enumerable object.

While it is possible to create your own query operators, the good news is that the .NET Framework already ships with a bevy of useful query operators. These query operators are referred to as the standard query operators and are one of the primary pieces of LINQ. The standard query operators include functionality for aggregating sequences of data, concatenating two sequences, converting sequences from one type to another, and splicing out a particular element from the enumeration. There are also standard query operators for generating new sequences, grouping and joining sequences, ordering the elements in sequences, filtering the data in a sequence, and partitioning the sequence.

All together, there are more than 40 standard query operators. This article explores some of the more germane ones, giving examples of the standard query operator in use and examining its underlying source code. There are also several demos included in the download available at the end of the article. Read on to learn more!

An Extensive Examination of LINQ: The Ins and Outs of Query Operators

mitchell@4guysfromrolla.com (Scott Mitchell) — Wed, 25 Mar 2009 00:00:00 GMT

As discussed in An Introduction to LINQ, LINQ is composed of three main components:

Language Extensions,
Standard Query Operators, and
LINQ Providers.

The previous two installments - Extension Methods, Implicitly Typed Variables, and Object Initializers and Lambda Expressions and Anonymous Types - explored the language extensions added to C# 3.0 and Visual Basic 9 that allow for LINQ's unique syntax. But these language extensions are merely syntactic sugar; they make it possible for developers to write more concise and readable code, but they don't actually perform any operations. The real workhorse of LINQ is the standard query operators.

LINQ's standard query operators are a collection of query operators. Query operators are methods that walk through a sequence of data and perform some task based on that data, and are implemented as extension methods on the IEnumerable interface. We've already seen a handful of standard query operators in use in previous installments, such as Where, Count, and Average. This article explores the inner workings of query operators, which is essential to understanding how LINQ's standard query operators work. Read on to learn more!

An Extensive Examination of LINQ: Lambda Expressions and Anonymous Types

mitchell@4guysfromrolla.com (Scott Mitchell) — Wed, 11 Mar 2009 00:00:00 GMT

The previous installment in this article series, Extension Methods, Implicitly Typed Variables, and Object Initializers, examined three new features to the C# 3.0 and Visual Basic 9 languages that allow for developers to use LINQ's standard query operators and providers to write SQL-like query syntax to work with common data stores. But extension methods, implicitly typed variables, and object initializers are only part of the story. Two additional language features - lambda expressions and anonymous types - are also essential ingredients in LINQ's unique syntax.

Recall that LINQ's standard query operators can work with any collection of data that implements the IEnumerable interface. Some of the standard query operators perform rather straightforward operations on the collection; the Count standard query operator, for instance, simply returns the number of elements in the collection. However, other standard query operators are more flexible, allowing the page developer to dictate how the operator will work. The Where standard query operator is such an example - it filters the elements in the collection based on a developer-specified filtering method. But just how can a developer "pass" a method to the Where standard query operator as an input parameter? The .NET Framework has long supported the notion of delegates, which are type-safe function pointers. However, the syntax for delegates is a little confusing and verbose. Lambda expressions offer a much more terse syntax for defining an anonymous method.

Anonymous types allow for inline types. In a nutshell, anonymous types let you clump a set of values together into a new type without having to first declare the type via a class. For example, imagine that you have a collection of Employee objects, where each Employee object has properties like Name, Salary, HireDate, and so on. You may want to apply some filtering condition on this collection using the Where standard query operator and then project the collection of Employee objects into a collection of objects that includes just the Name and Salary properties. With anonymous types this style of projection is straightforward and natural. Without anonymous types you would first have to create a class like EmployeeSimplified that contained only those two properties of interest.

This article provides an overview of lambda expressions and anonymous types. These two language enhancements, along with extension methods, implicitly typed variables, and object initializers, are what allows for LINQ's query syntax. Read on to learn more!

An Extensive Examination of LINQ: Extension Methods, Implicitly Typed Variables, and Object Initializers

mitchell@4guysfromrolla.com (Scott Mitchell) — Wed, 18 Feb 2009 00:00:00 GMT

One of the more substantive additions to the .NET Framework 3.5 and C# 3.0 and Visual Basic 9 languages was LINQ, a set of classes along with language enhancements that allow developers to use a common library and SQL-like query syntax to work with common data stores. The initial article in this series, An Introduction to LINQ, provided an overview of LINQ and its core pieces: the standard query operators, the language extensions that allow for LINQ's query syntax, and LINQ providers. We also looked at some simple LINQ examples using both the standard query operators and the query syntax.

LINQ's standard query operators - Select, Where, OrderBy, Average, and so on - can be used as if they were instance methods of any object that implements IEnumerable. For example, given a string array named FileNames we can determine how many strings in the array start with the letter "S" by using the Where and Count standard query operators like so:


string[] FileNames = { ... };

int count = FileNames.Where(name => name.StartsWith("S")).Count();

The Where method and Count methods look like they are members of the Array class. However, they are defined in the Enumerable class in the System.Linq namespace as extension methods on the IEnumerable interface. Also note the syntax that is used to specify the input parameter for the Where method: name => name.StartsWith("S"). This syntax is a lambda expression and provides a shorthand notation for developers to define a function.

This installment (and the next one) explore the language enhancements Microsoft made to C# 3.0 and Visual Basic 9 in more depth. (C# 3.0 and Visual Basic 9 are, at the time of this writing, the most recent versions of these two programming languages. They are the versions that were released with the .NET Framework 3.5 and Visual Basic 2008.) The key language enhancements that make LINQ possible include: extension methods; implicitly typed variables; object initializers; lambda expressions; and anonymous types. This article explores the first three of these language extensions; the latter two will be covered in the next installment. Read on to learn more!

An Extensive Examination of LINQ: An Introduction to LINQ

mitchell@4guysfromrolla.com (Scott Mitchell) — Wed, 11 Feb 2009 00:00:00 GMT

LINQ, or Language INtegrated Query, is set of classes added to the .NET Framework 3.5 along with language enhancements added to C# 3.0 and Visual Basic 9, the versions of the language that ship with Visual Studio 2008. LINQ adds a rich, standardized query syntax as a first-class citizen in .NET programming languages that allows developers to interact with any type of data.

Consider a typical data-driven application. There may be times when you are working with a database, displaying records or editing, inserting, and deleting data. Certain parts of the application may require retrieving certain elements from an XML file, or constructing an XML file based on user input. Or perhaps you have a collection of objects returned from a business object that you now want to work with by sorting them, computing the average value of a particular numeric property value, and displaying only those objects that meet a specified criteria. Prior to LINQ, working with each data source requires writing a different style of code. Moreover, working with external resources like data bases, XML files, and the like typically involves communicating with that external resource in some syntax specific to that resource. To retrieve data from a database you need to send it a string that contains the SQL query to execute; likewise, to work with a subset of XML elements in an XML document involves specifying an XPath expression in the form of a string. The idea is that using LINQ you can work with disparate data sources using a similar style without having to know a separate syntax for communicating with the data source (e.g., SQL or XPath) and without having to resort to passing opaque strings to external resources.

This article is the first in a series of articles that explores the goals of LINQ, its underpinnings, its syntax, and LINQ providers like LINQ to Objects, LINQ to XML, LINQ to SQL, and so forth. This inaugural article offers an overview of LINQ, looks at some simple examples of using the LINQ classes and syntax, and examines the core LINQ classes in the .NET Framework. Read on to learn more!