Tuesday, August 04, 2009

“I found $x very bizarre while learning powershell”

Coming from most other directions, powershell is truly bizarre. I started this a few years ago when I started learning PowerShell because I suspected that I would not be able to remember why powershell is so non-intuitive.

Confusing: Verb-Object?

I can’t be the only programmer who finds the syntax backwards. In almost every other language I’ve worked with, you have an object (or module) and call a method on it.

new EvilService().CreateRandomEvilPlan()

You can do that in powershell, but it’s not the powershell way.

Add-EvilPlan

The paradigms will get confused in most code, as anything that doesn’t have a commandlet uses the standard paradigm. You can access normal .NET objects (or com objects)

$proxy = New-WebServiceProxy -URI $uri -Class "DSP" -Namespace "Daptiv"

$proxy.

Whoa … Functions, and everything else, can be treated like a file.

Get-Item -Path function:

new-item -path function: -name csrss -options readonly -value {get-process csrss}

Everything in the output of this powershell method can be treated like a file system:

get-psdrive

For example:

get-item env:C*

dir env:C*

dir variable:ps*

A lot of functionality in powershell is exposed through providers with consistent syntax, revolving around items. This is a bit mind-blowing at first: I work with functions, the registry and certificates just like files? Then, it becomes fairly powerful.

Piping objects? Think reduce, not pipes

Shell script authors know pipes, but these are typically text or byte pipes, not object pipelines. Object pipelines are cool because 1) it’s really easy to distinguish one object from the next. No worry about terminators. 2) You can act on those objects, walk graphs within the pipeline (it’s not always a good idea, though).

Functional programming languages has the concept of reduce. I am happiest thinking of pipelines in functional programming terms.

Yeah … slices and easy list types!

While experimenting with python for operational purposes, I found the list handling superbly powerful. Defining a list of servers to act on, tuples to quickly group data, all were great. Some of these things can also be done in powershell. The operations aren’t made as easy in real languages, for fear of nasty performance problems working with long lists.

See help_about_array. Highlights.

$webservers= "webapp1","webapp2", “webapp3”

or strongly typed:

[string[]] $webservers= "webapp1","webapp2", “webapp3”

Slice it to get a part (webapp1, webapp2)

$webservers [0..1] = "webapp1","webapp2", “webapp3”

Use the plus operator (+) to combine two arrays

$dbservers= “db1”,“db2”

$allservers=$webservers + $dbservers

Dictionaries are as easy as I hoped. See help about_hash_tables

$lookup = @{coffee="bitter";apple="sweet";peanut="crunchy";banana="sweet"}

With a surprising access mechanism -- not $lookup(“peanut”), but

$lookup.peanut

Sorting and filtering lists and dictionaries is simpler and more powerful than most languages, because it ties into the powershell object pipeline.

$lookup = @{coffee="bitter";apple="sweet";peanut="crunchy";banana="sweet"}
$lookup.getenumerator() | sort-object -property key
$lookup.getenumerator() | where-object {$_.Value -eq "sweet"}

One liners can be more than one line

In summary, here is one “line” of powershell:

$x=(@("Verb-Object semantics",
    "everything is a drive",
    "piping objects",
    "slices and lists",
    "powershell line continuation",
    "string interpolation") `
    | Where-Object {!$_.Contains("lists")}); $y=[String]::Join(",",$x); "I found $y bizzare truly bizzare while learning powershell."

Thursday, July 30, 2009

Respect, don't trust: A little philosophy on the emotional life of the software team.

I haven't heard much talk about the emotional world of the software engineer, so I'll offer some late night ruminations on the topic.

The deliberate conflict between the engineers making software and quality assurance fascinates me. Here's why: While software engineers passionately try to make things work, QA engineers diligently work to prove that the software is horrible. If you put this behavior in another context there would be pain and strife. One person persistently pointing out someone else's flaws. How rude!

In a healthy team, this is not discord, it is work and sometimes even fun. It's beautiful to watch. The graceful dialectic offers some life lessons and gives me hope in human nature. We can be honest and frank, we can seek truth and scorn "white lies". We can do this respectfully, with passion, and without hurt.

Here is how software teams make this work.

Respect, don't trust 

QA never trusts development. A QA who trusts the developer would believe -- without empirical proof -- that the software works. That would be silly. QA's role is to aggressively and skeptically to prove the developers' work flawed. QA seeks to prove. While mistrusting, the QA team respect the developers intelligence, creativity and passion. Respect does not require trust in this relationship - It a relationship where mistrust and skepticism are possible, emotionally, because of the teams' strong mutual respect. 

Trust and faith are good for your wife and your religion. Anyone or anything less than these gods deserves respectful skepticism. 

Be fearless of mistakes - mistakes and corrections make the world work. 

There are few situations in life where products must be right the first time. This is true in assembly line manufacture and live virtuoso performances. Most of life is about mistakes and corrections. In a conversation, if we don't understand each other, we dialog and correct. If I don't catch a fish on the first cast, I can try again. Bikes constantly wobble a little as we balance from side to side.  

Productive software engineers don't fear mistakes. Engineers who fear mistakes hoard their work for weeks, keeping it to themselves. Finally, pride satisfied, they throw an overwhelming amount of change of the wall to QA. QA needs weeks to validate - the cycles take many times longer than they should and the product is invariably late, rough in the places that need to be smooth, and polished in places where it doesn't matter.

Software works better when the developer fearlessly checks in small pieces of work continually. A little due diligence - send to to quality assurance. The right things are fixed and fast. Back to the physical world analogies: A bicycle with small wobbles moves fast. The only bike that does not wobble is probably smoothly falling over. 

Back to respect: The developer can fearlessly check in because of the respect offered from QA. QA respects the daring of the developer showing her raw work to the team.

( The final product is of course flawless. Mistakes the customer sees are bad. I'm talking about the early life of software. Typically, we make hundreds or thousands of bad builds before we make the final release build. So, the customer sees the one "right" build, but the life of software development is a story of continual mistakes and corrections. )

A defect in software is not a defect of the author.

Defect systems offer a powerful emotional tool - we don't talk about the systems this way very often. I rarely hear, in software, QA walk up to an engineer and say, "You stupid lout. You made the CSS wrong and the now the border of this table clips the text."

Instead, the developers does there best to make something pretty. QA digs into it, fines a tiny flaw, and files a bug report into a bug database. The bugs get prioritized, assigned out and fixed. The separation means that the flaw never gets personal. The author can handle a statement like "There is a flaw in the software" much better than "You broke the software."

All software teams think defect systems are essential for organization - I argue they are also essential to avoid getting distracted by the emotions of making mistakes.

Tuesday, July 28, 2009

My Simple 3 Step IIS Setup System

This is my secret system for getting IIS to work on new computers (servers or desktops). Permissions, default identities and user interfaces change with each OS. I'm not going to write a book on all the things that can go wrong with IIS. I've got got 3 simple steps that shake out different types of problems very fast.

Audience: Software Engineers or QA Engineers don't want to reconfigure IIS more than a few times in a year. The folks in operations probably have a powershell script to do this.

Three steps and a tip

Starting point. New server? Installed W7 on your development machine and it isn't serving up your site any more? It's all the same problem.

Step 0: Start with a sanity check and Run Compmgmt.msc (computer management). Do you see Internet Information Services? Any suspicious icons?

Step 1: Drop a static web page into the site and browse to it.a For example: <html><body>Hello</body></html>

Step 2: Drop an a very simple ASPX page into the site and browse to it. Example: Response.Write("Hello"). My dynamic test page is a more complex than this, to report host header, virtual directory, and so forth.

Tip: Verify this site in multiple browsers. Even better, chose a browser with a “Private” mode. In some configurations (or mis-configurations) you might send NTLM credentials and the site impersonates your credentials, masking permissions problems.

Step X: What you don't want to do.

Don't debug basic IIS setup problems in a complex site. For example, daptiv will have a few dozen confounding factors: from special handlers to run of the mill database and storage configurations to several choices of authentication.

OK, here are some details.

But really, don't clutter up your mind by memorizing these. They are examples. IIS 6 configuration does not resemble IIS 7 configuration.

Step 0 problems - check the obvious

Are you missing an IIS component? It's easy to do. The components vary from release to release. On Windows 2003 x64 – installing SMTP requires CAB file hacking and you might have hacked it wrong.

By opening IIS Management, you verified that the metabase is readable. I've got a short list of known ways to corrupt the IIS MetaBase and longer list of unknowns. SysPrep, for example, can someimes mangles the machine keys IIS uses to secure its data. BTW, I don't know how to resolve metabase problems. The only method known to work without an overpaid IIS Consultant nearby is to uninstall IIS and reinstall.

Step 1 problems – static page

Is the web site running? Check IIS management console. The website might also be stopped because configuration isn't valid. Two sites on port 80 with the same url? Invalid application pool? These problems are easy because event log tells you the problem.

Permissions? If I can't see the static page on a new IIS instance, it is usually because of permissions at the virtual root. I get this often, as I’ll create webs at odd paths such e:\src\3\root\SecretProduct\www, and refuse to make IIS_IUSRS an admin on my box. Permissions should be easy: Add users one at a time in a sensible manner until it works. Remove them until it stops working. Now you know what needs access.

Account names will change between IIS, windows, and .NET versions. Typically I need to add permissions to the folder for:

  • Application pool identity – needs at least read access: My app pool usually runs as Network Service or a domain account.
  • The account used for anonymous access. (IIS_IUSRS or IUSR, IUSR_COMPUTERNAME, or something fancy)

Step 2 problems – ASPX fails

The ASP.NET extension might not be enabled. This is configured in entirely different places on IIS 6.0 (Web Service Extensions) and IIS 7 (ISAPI and CGI restrictions). It's disabled by default on some versions of Windows, ever after IIS is installed.

Of course, this is a good time to run aspnet_regiis.exe. If any identity (Application Pool) is not the default, use aspnet_regiis.exe -ga. Use aspnet_regiis -i with total abandon.

Summary

That's all I need to know to get most web servers running IIS. There is a basic troubleshooting principal here: start simple and build up facts.

Sunday, July 12, 2009

Czerneda - In the Company of Others

Another science fiction devoured. (Why review books?)

http://www.czerneda.com/novels.htm

The book:

Smart savvy herione (Earther Gail Smith) played off against a street smart heroes Pardell and Malley from the Thromberg station. The solar system and earth are quarantined against the vague Quill (what is it? the book leads off pointing out that humans are clearly the only intelligent life in the universe). Gail is from once side of the Quarantine, Malley and Pardell from the other. It all weaves nicely together. 

The experience:

Let's talk about Sunborn by Jeffrey Carver, because I picked up in the Company of Others after tossing aside Sunborn. Sunborn reminds me of a poorly written mockery of science fiction that one might see on a mediocre Saturday Morning cartoon trying to make fun of sci fi. Stilted characters. The author seeks to create a "rich" world by injecting new bizarre traits or situations every other page. He doesn't have the creativity to pull it off. The pages read like they were rolled off an assemlby line of the bizarre where the workers were having a bad day. Page one. Let's stamp something bizarre. OK, here's a telepathic link. Page two ... low on the bizarre. How about an intelligent sun? Page three ... oh pick a random page of P K Dick. An so on.

My reading of In the Company of Other's benefited greatly from the relief of coherence and quality it provided. I devoured it overnight, and had an unpleasantly tired Tuesday as a consequence. To add to the amusement of this book, at around 2AM I looked through the front window to see the (unabashed) neighbor boy sneak out with a gaggle of friends. Not large enough to be a posse and they look unaltered by sensibility reducing substances -- so it looked like good clean mischief. I just smiled with fond memories of those summer's of freedom that precede financial independence.

I suspected, as I read it, that I've read the sequel. The story tempo and characters seemed extremely familiar. I can't find a sequel, so it's entirely possible that I swallowed it whole before and forgot most of it. Overall, a perfect pulp sci fi experience.

Sunday, May 10, 2009

My Books to Read page: May 2009

Why?

Books to read for fun

Ursula Leguin

Ursula LeGuin makes me proud to spend most of my free time with nose in sci-fi speculative fiction books. Looks like she finally produced a new book!

* Lavina

Kage Baker

Where can I find the Children of Company novels, in order? Is there an order?

Patrick Rothfuss

Name of the Wind hooked me. Where is the next one? Wake me up when it's done.

David Gerrold

Maybe Rothfuss will finish his books before David Gerrold finishes the war of the Chtorr. I think I grew out of that series, and then grew up enough that I could read it anyway. Damn artistes.

Serious Reading

Trigonometry for dummies (in progress)

Backlog of serious reading

  • Either "7 Habits of Highly Effective People" or "Getting Things Done: The art of stress-free productivity".
  • Mathematical Methods in the Physical Sciences: Need to keep the math fresh.
  • NHibernate in action. (In progress.) NHibernate only seems necessary to me because other people think it's necessary. I've never had a problem it purports to solve. The unit of work and caching in it is decent, but too tied into other crap to be be useful. Is there any ROI? Have to learn a clunkly framework that will be legacy in a few years.
  • I'll probably find a good ASP.NET MVC book and read it before I force myself to dig into NHibernate.

Read and digesting

  • Domain Driven Design, Eric Evans: This is the classic
  • Applying Domain-Driven Design and Patterns, Nilson: A good folks read, which just enough to disagree with.

Books on my Safari account - used for occasional skimming

  • Clean Code (great)
  • IronPython in Action: Just started
  • C# 3.0 in a nutshell: Perhaps Worth buying?
  • Deploying .NET applications: Disappointing.

Applying Domain-Driven Design and Patterns, Nillson (read ... drafting review)

Serious reading, looking for

  • Books on running and exercise that are not lame.
  • Books on finance.
  • Behavior Driven Design.
  • Content management, collaboration, OpenID

Hello, World in msbuild

I haven't blogged in a while. Here is a msbuild "Hello, World" for you.

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build;Test" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<WhoToGreet>World</WhoToGreet>
</PropertyGroup>

<Target Name="Hello">
<Message Text="Hello, $(WhoToGreet)" Importance="high" ></Message>
</Target>

<!-- A build target is required. -->
<Target Name="Build" DependsOnTargets="Hello"/>
<!-- A test target is required. -->
<Target Name="Test"/>
</Project>


Execute as : msbuild helloworld.proj



or : msbuild helloworld.proj /p:WhoToGreet="Moon"