Thursday @ 2:31 am
- from Maui, Hawaii
If you're doing AJAX callbacks to the server with POST data from ASP.NET pages you might be inclined to do something like this:
var post = $("#form1").serialize();
$.post("MethodCallback.aspx?Callback=ListPanel",
post,
function(result) { alert(result) });
This works fine, but it will pick up all of your ViewState and EventValidation making the post back to the server a heck of a lot bigger than it should be especially if you have a page where ViewState is heavily used. Now I tend to turn off ViewState on most of my pages but you still get at least some ViewState and EventValidation content on the page and in most Ajax scenarios sending that to the server is just wasted bandwidth.
Fortunately it's quite easy to exclude ViewState et al. with code like this:
var res = $("#form1")
.find("input,textarea,select,hidden")
.not("#__VIEWSTATE,#__EVENTVALIDATION")
.serialize();
$.post("MethodCallback.aspx?Callback=ListPanel",
res,
function(result) { alert(result) });
This is a little more verbose, but necessary in order to be able to individually reference each of the child input elements. The #Form1 selector is optional - if you only have a single form or if you don't care about posting all form vars regardless of belonging form, you can just request all input,select,textarea,select and hidden elements.
The .not() function basically filters the result set of input elements and so doesn't send up ViewState and Event validation. If you do this a lot a small generic plug-in might be useful:
$.fn.serializeNoViewState = function()
{
return this.find("input,textarea,select,hidden")
.not("[type=hidden][name^=__]")
.serialize();
}
which then allows you to do:
var res = $("#form1").serializeNoViewState();
or even:
var res = $().serializeNoViewState();
to select all form data.
Reducing ViewState and avoiding it from being sent back and forth between client and server can drastically improve AJAX performance so it's a good idea to be concious of what goes over the wire on those presumably 'small' AJAX requests. Removing it from the client POST is one quick way to reduce bandwidth.
Note that if you plan on having the page actually go through a normal 'Postback' like stage and you want to capture events during an AJAX callback you can also post back ViewState et al and this can in many cases work, giving you access to POST data on form fields (ie. referencing like this.txtName.Text rather than explicitly calling Request.Form[]). But it's best not to rely on this mechanism as there are lots of little quirks and gotchas. Ajax callbacks are best handled as independently as possible - preferably not back to the same page or a service to avoid any conflicts with ViewState and EventValidation. Otherwise turning ViewState and EventValidation off will help prevent page errors on callbacks or a full Postback following many AJAX updates. This is no trivial matter so you have to understand what's being updated on the client and what will actually make it back to the server (other than form fields). This is why it's best to minimize reliance on Viewstate so that Viewstate corruption and EventValidation won't be a problem.
Encoding POST data via Object Maps
Since we're talking about jQuery POST data encoding - note that you can also use the $.serialize() function to turn any flat object map into POST data. So you can easily create POST data like this:
var post = { name: "Rick",
company: "West Wind",
lastOn: new Date(),
count: 30
};
$.post("MethodCallback.aspx?CallbackHandle=ListUpdate",
post,
function(result) { alert(result) });
which in turn turns all the map values - name, company, lastOn, count - into POST variables that are sent to the server URL encoded. Most places where a 'data' parameter is expected with POST data in jQuery you can either pass a string (as in the first examples above) or an object map as shown here. This is a nice and easy way to create POST data without having to manually worry about POST encoding it.
Behind the scenes jQuery uses the static $.serialize(data) function which also provides this functionality to your code and lets you turn any object map into a POST string explicitly.
var post = { name: "Rick", email: "rick@morecowbell.com" };
var poststring = $.serialize(post);
Very useful if you're interacting with REST based applications that work purely on POST data rather than JSON encoded content.
Lots of flexibilty there and it's easy to integrate with ASP.NET.
Wednesday @ 1:22 pm
- from Maui, Hawaii
I've been thinking about replacing the JSON serializer in my internal codebase for some time and finally put a little effort into allowing JSON.NET from James Newton King to be plugged into my Ajax framework as a replaceable engine. My own parser has served me well, but JSON.NET is a much cleaner design and more flexible especially when dealing with extensibility. There are also a few nice features like the ability to pretty-format the generated JSON for debugging purposes which is immensely helpful when sending data to the client during development.
Although I'm keeping my original parser class (including original basic functionality) JSON.NET can now be optionally plugged in. So why not just replace the parser altogether? As it turns out I've been experimenting with different JSON serializer/deserializers and being able to keep a 'high level' wrapper object into which each of these parsers can be plugged into has been a great time saver as the interface that various application components use doesn't change one bit which is nice indeed.
Anyway, here's the basic code to provide JSON encoding and deserialization in a simple wrapper functions:
public string Serialize(object value)
{
Type type = value.GetType();
Newtonsoft.Json.JsonSerializer json = new Newtonsoft.Json.JsonSerializer();
json.NullValueHandling = NullValueHandling.Ignore;
json.ObjectCreationHandling = Newtonsoft.Json.ObjectCreationHandling.Replace;
json.MissingMemberHandling = Newtonsoft.Json.MissingMemberHandling.Ignore;
json.ReferenceLoopHandling = ReferenceLoopHandling.Ignore;
if (type == typeof(DataRow))
json.Converters.Add(new DataRowConverter());
else if(type == typeof(DataTable))
json.Converters.Add(new DataTableConverter());
else if (type == typeof(DataSet))
json.Converters.Add(new DataSetConverter());
StringWriter sw = new StringWriter();
Newtonsoft.Json.JsonTextWriter writer = new JsonTextWriter(sw);
if (this.FormatJsonOutput)
writer.Formatting = Formatting.Indented;
else
writer.Formatting = Formatting.None;
writer.QuoteChar = '"';
json.Serialize(writer, value);
string output = sw.ToString();
writer.Close();
sw.Close();
return output;
}
public object Deserialize(string jsonText, Type valueType)
{
Newtonsoft.Json.JsonSerializer json = new Newtonsoft.Json.JsonSerializer();
json.NullValueHandling = Newtonsoft.Json.NullValueHandling.Ignore;
json.ObjectCreationHandling = Newtonsoft.Json.ObjectCreationHandling.Replace;
json.MissingMemberHandling = Newtonsoft.Json.MissingMemberHandling.Ignore;
json.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore;
StringReader sr = new StringReader(jsonText);
Newtonsoft.Json.JsonTextReader reader = new JsonTextReader(sr);
object result = json.Deserialize(reader, valueType);
reader.Close();
return result;
}
One of the nice things about the JSON.NET parser is that there are a tons of configuration options that allow you to customize how JSON parsing occurs. You can see some of the options above for things like null value handling and managing how deep recursive loops should be handled for example.
JSON.NET also has a simplified JavaScriptConvert class that has much simpler serialization and deserialization methods, but this wrapper doesn't have access to the configuration options, so creating a custom wrapper that does exactly what I need certainly helps. As you can see above the parser has a ton of control over how many parsing aspects are handled which is really cool.
The really nice thing though - and the thing that really is missing in my parser - is extensibility. You can create custom Converters that can be plugged into the parsing pipeline to serialize and optionally deserialize custom types. Any JSON parser is likely to do well with most common types, but there may also be custom types or structures that are not supported or not well supported. For example, the ADO.NET objects, or a less obvious problem of how IDictionary types are handled (JSON.NET only supports string keys and then only those that result in valid JavaScript property/map names).
Luckily you can create your own converters, which is a common way of extensibility. The ASP.NET JavaScriptSerializer also supports extension via Converters. So, my immediate need before I could use the JSON.NET parser for a couple of old apps was to create a parser that can serialize JSON from Datatables.
Here's are a few converters that can create DataSet/DataTable and DataRow JSON with JSON.NET and turns them into simple value arrays (ie Dataset.Tables[].Rows[]). Note the converters are only for serialization not deserialization:
/// <summary>
/// Converts a <see cref="DataRow"/> object to and from JSON.
/// </summary>
public class DataRowConverter : JsonConverter
{
/// <summary>
/// Writes the JSON representation of the object.
/// </summary>
/// <param name="writer">The <see cref="JsonWriter"/> to write to.</param>
/// <param name="value">The value.</param>
public override void WriteJson(JsonWriter writer, object dataRow)
{
DataRow row = dataRow as DataRow;
// *** HACK: need to use root serializer to write the column value
// should be fixed in next ver of JSON.NET with writer.Serialize(object)
JsonSerializer ser = new JsonSerializer();
writer.WriteStartObject();
foreach (DataColumn column in row.Table.Columns)
{
writer.WritePropertyName(column.ColumnName);
ser.Serialize(writer,row[column]);
}
writer.WriteEndObject();
}
/// <summary>
/// Determines whether this instance can convert the specified value type.
/// </summary>
/// <param name="valueType">Type of the value.</param>
/// <returns>
/// <c>true</c> if this instance can convert the specified value type; otherwise, <c>false</c>.
/// </returns>
public override bool CanConvert(Type valueType)
{
return typeof(DataRow).IsAssignableFrom(valueType);
}
/// <summary>
/// Reads the JSON representation of the object.
/// </summary>
/// <param name="reader">The <see cref="JsonReader"/> to read from.</param>
/// <param name="objectType">Type of the object.</param>
/// <returns>The object value.</returns>
public override object ReadJson(JsonReader reader, Type objectType)
{
throw new NotImplementedException();
}
}
/// <summary>
/// Converts a DataTable to JSON. Note no support for deserialization
/// </summary>
public class DataTableConverter : JsonConverter
{
/// <summary>
/// Writes the JSON representation of the object.
/// </summary>
/// <param name="writer">The <see cref="JsonWriter"/> to write to.</param>
/// <param name="value">The value.</param>
public override void WriteJson(JsonWriter writer, object dataTable)
{
DataTable table = dataTable as DataTable;
DataRowConverter converter = new DataRowConverter();
writer.WriteStartObject();
writer.WritePropertyName("Rows");
writer.WriteStartArray();
foreach (DataRow row in table.Rows)
{
converter.WriteJson(writer, row);
}
writer.WriteEndArray();
writer.WriteEndObject();
}
/// <summary>
/// Determines whether this instance can convert the specified value type.
/// </summary>
/// <param name="valueType">Type of the value.</param>
/// <returns>
/// <c>true</c> if this instance can convert the specified value type; otherwise, <c>false</c>.
/// </returns>
public override bool CanConvert(Type valueType)
{
return typeof(DataTable).IsAssignableFrom(valueType);
}
/// <summary>
/// Reads the JSON representation of the object.
/// </summary>
/// <param name="reader">The <see cref="JsonReader"/> to read from.</param>
/// <param name="objectType">Type of the object.</param>
/// <returns>The object value.</returns>
public override object ReadJson(JsonReader reader, Type objectType)
{
throw new NotImplementedException();
}
}
/// <summary>
/// Converts a <see cref="DataSet"/> object to JSON. No support for reading.
/// </summary>
public class DataSetConverter : JsonConverter
{
/// <summary>
/// Writes the JSON representation of the object.
/// </summary>
/// <param name="writer">The <see cref="JsonWriter"/> to write to.</param>
/// <param name="value">The value.</param>
public override void WriteJson(JsonWriter writer, object dataset)
{
DataSet dataSet = dataset as DataSet;
DataTableConverter converter = new DataTableConverter();
writer.WriteStartObject();
writer.WritePropertyName("Tables");
writer.WriteStartArray();
foreach (DataTable table in dataSet.Tables)
{
converter.WriteJson(writer, table);
}
writer.WriteEndArray();
writer.WriteEndObject();
}
/// <summary>
/// Determines whether this instance can convert the specified value type.
/// </summary>
/// <param name="valueType">Type of the value.</param>
/// <returns>
/// <c>true</c> if this instance can convert the specified value type; otherwise, <c>false</c>.
/// </returns>
public override bool CanConvert(Type valueType)
{
return typeof(DataSet).IsAssignableFrom(valueType);
}
/// <summary>
/// Reads the JSON representation of the object.
/// </summary>
/// <param name="reader">The <see cref="JsonReader"/> to read from.</param>
/// <param name="objectType">Type of the object.</param>
/// <returns>The object value.</returns>
public override object ReadJson(JsonReader reader, Type objectType)
{
throw new NotImplementedException();
}
}
Maybe somebody else will find this useful as well. The serialization is one way only because I personally don't have any need for two-way and deserialization of these objects is tricky because deserialization requires a type to match properties to values in the parsed JSON and well these ADO structures don't have any type format.
Anyway JSON.NET is really nice because it's highly configurable and actively developed. It's also open code so you can look through it and see how its done and tweak if necessary. One thing though - it appears it is a bit slower than other solutions. I've found JSON.NET 2.0 to be more than twice as slow as other solutions, and JSON.NET 3.0 as much as 6 to 7 times slower in some informal iteration tests built into my test suite (I noticed this because tests were slowing down drastically on the perf loop). While processing times are still very small (7 seconds for 10,000 serializations w/ JN 3.0) it still rankles a bit when its considerably slower than other solutions.
DataSet/DataTable/DataRow Serialization in JavaScriptSerializer
The stock JavaScriptSerializer that ships with System.Web.Extensions as part of .NET 3.5 also doesn't directly support ADO.NET objects. Some time ago there was a Converter provided in the Futures package (might still be but I haven't checked) but I never actually used it because the Futures assembly is just too much in flux and you never know what sticks around and will get axed.
Luckily it's also pretty easy to create custom converters for JavaScriptSerializer and so it's easy to create DataSet/Table/Row converters that you can add to your own apps without having to rely on the futures DLL. Here's that same implementation for the JavaScriptSerializer:
internal class WebExtensionsJavaScriptSerializer : JSONSerializerBase, IJSONSerializer
{
public WebExtensionsJavaScriptSerializer(JSONSerializer serializer) : base(serializer)
{}
public string Serialize(object value)
{
JavaScriptSerializer ser = new JavaScriptSerializer();
List<JavaScriptConverter> converters = new List<JavaScriptConverter>();
if (value != null)
{
Type type = value.GetType();
if (type == typeof(DataTable) || type == typeof(DataRow) || type == typeof(DataSet))
{
converters.Add(new WebExtensionsDataRowConverter());
converters.Add(new WebExtensionsDataTableConverter());
converters.Add(new WebExtensionsDataSetConverter());
}
if (converters.Count > 0)
ser.RegisterConverters(converters);
}
return = ser.Serialize(value);
}
public object Deserialize(string jsonText, Type valueType)
{
// *** Have to use Reflection with a 'dynamic' non constant type instance
JavaScriptSerializer ser = new JavaScriptSerializer();
object result = ser.GetType()
.GetMethod("Deserialize")
.MakeGenericMethod(valueType)
.Invoke(ser, new object[1] { jsonText });
return result;
}
}
internal class WebExtensionsDataTableConverter : JavaScriptConverter
{
public override IEnumerable<Type> SupportedTypes
{
get { return new Type[] {typeof (DataTable)}; }
}
public override object Deserialize(IDictionary<string, object> dictionary, Type type,
JavaScriptSerializer serializer)
{
throw new NotImplementedException();
}
public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer)
{
DataTable table = obj as DataTable;
// *** result 'object'
Dictionary<string, object> result = new Dictionary<string, object>();
if (table != null)
{
// *** We'll represent rows as an array/listType
List<object> rows = new List<object>();
foreach (DataRow row in table.Rows)
{
rows.Add(row); // Rely on DataRowConverter to handle
}
result["Rows"] = rows;
return result;
}
return new Dictionary<string, object>();
}
}
internal class WebExtensionsDataRowConverter : JavaScriptConverter
{
public override IEnumerable<Type> SupportedTypes
{
get { return new Type[] { typeof(DataRow) }; }
}
public override object Deserialize(IDictionary<string, object> dictionary, Type type,
JavaScriptSerializer serializer)
{
throw new NotImplementedException();
}
public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer)
{
DataRow dataRow = obj as DataRow;
Dictionary<string, object> propValues = new Dictionary<string, object>();
if (dataRow != null)
{
foreach (DataColumn dc in dataRow.Table.Columns)
{
propValues.Add( dc.ColumnName, dataRow[dc]);
}
}
return propValues;
}
}
internal class WebExtensionsDataSetConverter : JavaScriptConverter
{
public override IEnumerable<Type> SupportedTypes
{
get { return new Type[] { typeof(DataSet) }; }
}
public override object Deserialize(IDictionary<string, object> dictionary, Type type,
JavaScriptSerializer serializer)
{
throw new NotImplementedException();
}
public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer)
{
DataSet dataSet = obj as DataSet;
Dictionary<string, object> tables = new Dictionary<string, object>();
if (dataSet != null)
{
foreach (DataTable dt in dataSet.Tables)
{
tables.Add(dt.TableName, dt);
}
}
return tables;
}
}
Implementing a custom converter involves creating dictionaries of property name and value pairs with the abillity to create nested properties by specifying another dictionary as a value. This approach is makes it pretty easy to create custom serialization formats...
I know I'm not using DataSets much anymore these days but with older code still sitting around that does these converters are coming in handy. Maybe this will prove useful to some of you in a similar situation.
Wednesday @ 1:54 am
- from Maui, Hawaii
If you're working with dates that have been sent from the client encoded in JSON format, and you're using a JSON parser to handle the parsing for you make sure you deal carefully with date values. The various JSON serializers out there handle date formatting slightly differently and this can make a subtle difference that you might not immediately notice when dates are retrieved from the JSON.
Specifically the issue is that Microsoft JSON deserializers - JavaScriptSerializer and DataContractJsonSerializer - return dates as Utc style dates.
This can result in what appears to be strange results when you run code that handles both encoding and decoding:
[TestMethod]
public void DateTest()
{
DateTime date = DateTime.Now.AddDays(-10);
JavaScriptSerializer sr = new JavaScriptSerializer();
string json = sr.Serialize(date);
DateTime retrieved = sr.Deserialize<DateTime>(json);
Assert.AreEqual(retrieved, date, "Date failed to deserialize");
}
This code will fail because the dates are off by the timezone offset difference between the Utc date and the local time. So to be safe it's a good idea to always convert the date to the proper date kind before using it to be sure you have the right date context. The following should work reliably:
Assert.AreEqual(retrieved.ToLocalTime(),date, "Date failed to deserialize");
or using .ToUtcTime() if you expect the date to be in Utc. Explicitly forcing the date to a specific DateKind is a good idea just to make sure if you end up switching parsers in the future.
The reasoning behind this makes some sense - the JSON encoded value is actually a tick count that is given in UTC and since there's no information on what kind of date you are dealing with the date returned is always in Utc. Of course the .NET code can make a decision to return the date in any format it chooses but the parsed raw date tends to be in Utc first.
Still returning Utc seems very inconsistent - what API (except explicit Utc related/named methods) returns a UTC date? The above behavior might be very unexpected and worse often go undetected for some time until somebody carefully checks captured data because the date values will be close to right but just off by the timezone offset.
In any case it's probably a good idea to always force dates returned from JSON to the appropriate date kind just to be safe.
BTW, I'm happy to see Microsoft removed the 'Deprecated' tag from the System.Web.Extensions.JavaScriptSerializer class. Although DataContractJsonSerializer was supposed to supercede DataContractSerializer, the interface and pre-requisites for JSON serialization tend to be easier and more flexible using the JavaScriptSerializer.
August 31, 2008 @ 2:12 pm
- from Maui, Hawaii
FireFox 3.0 apparently is much more aggressive in caching content than previous versions of FireFox. While this is good for performance in many situations I've noticed that this is seriously becoming a problem in some of my applications where support files like scripts and CSS files get updated and FireFox doesn't recognize those changes. The problem appears to be that FireFox is not sending the required 304 request to check for last-modified headers that are typically used for static files.
The latest example is my (ancient) Message Board site which I recently updated by reducing the HTML and externalizing most of the inline styling into an external CSS file that previously existed. Today I'm waking up to 10 messages from visitors that the the formatting of the message board is completely mangled. Apparently the CSS file changes are not being picked up by FireFox 3.0 and so users see the new dynamic message board pages, rendered against the old CSS styles cached in the browser. Thankfully I did this over the weekend and so the cacophony of disgruntled visitors was only minor.
The updated CSS file is there on the server. IE, Opera and Safari and even FF 2 all work fine - only FF3 fails to refresh the CSS for clients because apparently it's not detecting the last-modified date for the static resource. Even pressing Refresh doesn't fix this and even FireFox's 'hard' Shift-Refresh doesn't do the trick if the CSS is already cached.
Part of the problem here is that the app is frames based. Shift-Refresh should in theory force a refresh of all resources (and it does usually), but in the case of this frames based layout it doesn't. The only way to force the CSS to refresh that I found is to Ctrl-Click one of the links and force into a new frame, then force a Shift-Refresh in the new window. Intuitive, eh? Oh, or you can clear the browser cache! Yeah - that's what I tell 'em! <g>
But it's not only this CSS that suffers. I've also had issues with AJAX related static JavaScript files served from the server that also didn't work with the same caching issues. This is even more insidious because you can't see the problem except that potentially your app will fail now because the application's client application data is not up to date.
Again only a Shift-Refresh usually solves this problem. While this is fairly painless to do in the dev environment (it's become a habit already to Ctrl-Click) this is a big problem in production environments since you certainly don't want to tell your visitors to 'just Ctl-Click and things will be cool'.
It appears that the problem is vaguely related to this post:
With Firefox 1 and 2, “Cache-control: no-cache” accomplished this goal. I’ve spend the last several hours trying to figure out how to prevent Firefox 3 from caching my pages, with no luck. It completely ignores “Cache-control: no-store, no-cache, must-revalidate” and every combination of values for the “Expired:” header that I can think of, in blatant violation of RFC 2616 (14.9 and 14.21).
This post also mentions how you can modify your cache settings. While this works this is hardly a solution for end users.
Last-Modified Headers not working properly in FF3
The issue described in the post is in regards to caching, which is a little different actually than what I'm experiencing here, which has to do with the last-modified header that static files use. Web Servers serve static files (like CSS and scripts) from disk and they tend to serve this data with a last-modified header which the browser is supposed to respect. When requesting data again for a page that when originally loaded the browser is supposed to fire a 304 request to see if the content has changed.
Here's what the 304 response looks like in Internet Explorer with Fiddler enabled. IE fires the hit to the CSS file every time as a 304 request which looks like this:
GET /wwThreads/wwthreads.css HTTP/1.1
Accept: */*
Referer: http://www.west-wind.com/wwThreads/head.htm
Accept-Language: en-us
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; FDM; .NET CLR 3.5.21022; .NET CLR 3.5.30729; .NET CLR 3.0.30618; .NET CLR 1.1.4322)
UA-CPU: x86
Accept-Encoding: gzip, deflate
If-Modified-Since: Sun, 31 Aug 2008 09:37:26 GMT
If-None-Match: "20d7b72d4dbc91:0"
Connection: Keep-Alive
Host: www.west-wind.com
HTTP/1.1 304 Not Modified
Last-Modified: Sun, 31 Aug 2008 09:37:26 GMT
ETag: "20d7b72d4dbc91:0"
Accept-Ranges: bytes
Server: Microsoft-IIS/7.0
X-Powered-By: ASP.NET
Date: Sun, 31 Aug 2008 20:04:11 GMT
If the content were modified the server would respond with a full 200 HTTP response. So if I go in and modify my CSS file and hit the page again, IE will properly detect the change.
FireFox however, doesn't appear to fire the 304 at all. I've hooked up Firebug and the CSS files don't show up at all. Not trusting the FireBug network tracing I also hooked up FireFox to Fiddler and lo and behold it too shows that FireFox is not firing the 304's to check for updated files. When loading pages - even with Ctl-Clicking - I don't see the CSS files loaded which explains why the pages render all mangled. In my frames page even browser Refresh or even a Shift-Refresh doesn't cause the frames to reload their content or fire 304s to check for updates - all the static content is simply not accessed at all.
For example, if I start up FireFox fresh and navigate to my site and hit one link here's what Fiddler shows:
Notice there are no other requests to any support resources - images, CSS etc. No checks for 304's. Here's that same trace with Internet Explorer:
:
and you'll notice all the 304 hits to verify resources. Now using Shift-Refresh in Firefox will force FireFox to refresh all those links explicitly and load them, but this isn't the expected behavior. The whole point of last-modified and 304s is to be able to detect changes to static resources and apparently FireFox doesn't play along here.
Frames
Ok, frames are so passé but as I mentioned the app I'm actually dealing with uses frames and the problem there is that a Shift-Refresh on the parent frame isn't enough to force the individual frames to refresh. In fact, I'm not really sure how to force FireFox to refresh an individual frame. The only way I've been able to do this is to load one of the frames by shift clicking into a separate window and then doing a shift-Refresh to force the stylesheet to reload. After that all's well because that updates the cache globally.
Refresh should be Refresh
What's worse though is that in FireFox 3's default configuration even clicking the Refresh button will not force 304s. Apparently the Refresh button behavior has been changed to only re-run the last request EXACTLY in the same mode it was run. So if you did a Shift-Refresh on the last hit, it will repeat that behavior. If you did a plain link click that will replay exactly and so on. There's no longer an explicit reloading of resources which is now delegated to Shift-Refresh and which no 'typical' user will think of.
The FireFox Refresh button's Refresh behavior is pretty lame. Having an easy way to force pages and all related content to reload is key in a browser environment where the underlying resources may frequently change on the server. Between the refresh and last-modified issues described FireFox 3.x is becoming a bit shaky for the stability, especially for applications that are rich in client side styling and script behavior. It's not uncommon for my apps to update CSS or script content several times a day and there's now a good possibility that these changes may not match changes in the underlying application that serves non-static content. This has bit me quite frequently now during development when refreshing pages and expecting to see update content only to find that things like Script files have not refreshed and a few times in production with customers complaining.
One temporary solution might be (thanks to Steve Black for the hint) to force the CSS references in pages to refresh by changing the URL slightly. Like
<link href="wwThreads.css?i=1" rel="stylesheet" type="text/css"/>
The additional querystring will force even FireFox to refresh the data. But that's one heck of a hack. But I may have to resort to this because nothing else works reliably at this point.
So... has anybody else run into this in their applications? In searching around the Web it doesn't appear that there's much talk about this issue - I didn't find anything related to last-modified and a few discussions - which are related - to caching and back button and refresh behavior. It looks like Firefox is trying to be too agressive in caching. Sure, caching is cool for performance, but not if it's at the cost of stale content.
August 28, 2008 @ 3:05 am
- from Maui, Hawaii
Ran into another fun little problem a few days ago. Working on my root Web site which is rather large and contains a huge number of sub-webs. The root site is very light in terms of ASP.NET functionality used - primarily stuff like cookie tracking and logging tasks, serving banners etc and a few utility applications. Most of the heavy lifting on the site and 'real' applications are managed in virtual directories that handle real processing.
So yesterday over the last few days I've been building a small utility form that hooks up to Html Help Builder and allows generating documentation online for demonstration. This page lives in a directory off the root Web site, but it's part of the Root Web application and so runs under the root site. At some point I needed to debug the code and so I start the debugger on this large Root web and...
Well it looks like the debugger is starting up, but in fact it's just starting and then immediately aborting. There's no error message, no output window message, but IIS or the internal Web Server simply fail to attach the debugger and so while the app runs just fine, there's no debugging.
My first thought was that this is caused because it's a root Web and the site is very large. There are over 10,000 files that are part of the root web and singnificantly more when counting the child virtuals that live underneath the root. Surprisingly VS 2008 SP1 deals with this sizably Web pretty well in terms of speed of bringing up the solution - size is apparently not the problem.
Alas, it turns out the problem has nothing to do with size, but rather that I have a <location> tag in my web.config that pretty much wraps the entire web.config to prevent the root web settings from trickling down into child Web sites. Specifically I don't want authentication and handlers/modules be required in child virtuals which otherwise would require every child web to either explicitly remove handlers and modules or add them to the virtual's bin folders which is shitty to say the least.
My workaround for this is to use a <location> tag
<?xml version="1.0"?>
<configuration>
<location inheritInChildApplications="false">
<system.web>
<compilation debug="true">
<assemblies>
<add assembly="System.DirectoryServices, Version=2.0.0.0, Culture=neutral, PublicKeyToken=B03F5F7F11D50A3A" />
</assemblies>
</compilation>
<pages>
<namespaces>
<add namespace="Westwind.Tools"/>
<add namespace="Westwind.Web.Controls"/>
<add namespace="Westwind.Web.Banners"/>
</namespaces>
</pages>
<trust level="Full"/>
<authentication mode="Windows"/>
<customErrors mode="RemoteOnly" defaultRedirect="GenericErrorPage.htm">
<error statusCode="403" redirect="NoAccess.htm"/>
<error statusCode="404" redirect="FileNotFound.htm"/>
</customErrors>
</system.web>
<system.webServer>
<handlers>
<add verb="GET"
name="wwBanner"
path="wwBanner.ashx"
type="Westwind.Web.Banners.BannerHandler, Westwind.Web.Controls" />
</handlers>
<validation validateIntegratedModeConfiguration="false"/>
</system.webServer>
</location>
</configuration>
The inheritInChildApplications="false" is quite useful and prevents any of the settings in the configuration to riple down to child virtuals. This attribute can be applied to many section headers, but if you want a whole slew of tags not to propagate down the <location> tag is a great place to apply this attribute because it effectively encapsulates all data beneath it.
Unfortunately it looks like this configuration is also the cause that Visual Studio fails to start the debug. Removing the <location> block allows me to debug just fine.
Specifically it's the compilation section that must be outside of the <location> block. If I move the <compilation> section outside of the location block I can also start debugging. The following also works:
<?xml version="1.0"?>
<configuration>
<system.web>
<compilation debug="true">
<assemblies>
<add assembly="System.DirectoryServices, Version=2.0.0.0, Culture=neutral, PublicKeyToken=B03F5F7F11D50A3A" />
</assemblies>
</compilation>
</system.web>
<location inheritInChildApplications="false">
...
Depending on what you have inside the <assemblies> section of the compilation tag this may or may not be acceptable as referenced assemblies here will get loaded when the app starts and so can slow down load times for your child virtual AppDomains.
I suppose it's not terrible problem because this is likely only going to be a problem only in debugging scenarios. It's not a big deal, but it's yet another mysterious cause for when debugging doesn't work, but hopefully this will help out those of you searching for a solution to a similar problem.
August 27, 2008 @ 7:04 pm
- from Maui, Hawaii
So I just lost all of my project templates in Web projects. I've been working on a small utility page on my site that generates help documentation on the fly. I've been working happily along on this site when all of a sudden when I needed to add a config file to control debugging and authentication on this page/app.
could not find a part of the path 'C:\programs\vs2008\ Common 7\IDE\ItemTemplatesCache\Web\CSharp\1033\webConfig.zip\webConfig.vstemplate'.
A quick check in the above path oddly shows that the webConfig.zip file exists in this location and the content of the file is fine as well. Trying to add a Web page or a class in the App_code folder too fails with the same error.
I have no idea why this happened just now, I didn't install anything in VS in the last couple of days so it certainly seems like it 'just broke'.
Searched around a bit and found a note on the ASP.NET forums by Mikhail Arkhipov with a hint to re-register the Visual Studio templates. To do this run:
DevEnv.exe /installvstemplates
and life is good again.
August 26, 2008 @ 6:18 am
- from Maui, Hawaii
I'm calling a COM object from managed code that's returning a binary response, which is returned as a SafeArray of bytes from the COM server. My managed code uses Reflection to retrieve the result from the COM Server like this:
// Results in a binary (SafeArray) response from COM object
// .ToString shows: System.Byte[*] as the type
object zipContent = wwUtils.CallMethodCom(this.HelpBuilder,
"CreateHelpZip", "DOTNETASSEMBLY", "TestProject",
"MSDN",
@"C:\projects2008\Westwind.Tools\bin\Debug\Westwind.Tools.dll", 50);
// *** This fails
byte[] content = (byte[])zipContent;
CallMethodCom simply is a wrapper around Type.Invoke() so it's basic Reflection call. The method call works fine and it appears to return the expected result at least from what I can see in the debugger. But in code
What's interesting is that when I step through this code the code comes back properly. I get an object that according to the debugger contains a valid byte[] array:
But the code that casts the result fails. I get an error that declares: "Unable to cast object of type 'System.Byte[*]' to type 'System.Byte[]".
Haeh? System.Byte[*]? What the heck is that? A 'fixed width' byte array? What's really confusing here is that the debugger shows the object properly as byte[]. I can even cast to byte[] in the Immediate Window and get at the byte[] properties like Length. So it's working but somehow the generated C# runtime code fails to cast the byte array.
[Updated based on Comments from Christof and Kevin]
It turns out there is a way to reference the SafeArray in .NET which is as a non typed, one-based array. You can interact with this array only using GetValue(),SetValue() or by copying it out using CopyTo(), which effectively means the only way to retrieve the data is to copy it into a byte array or else write it out one byte at a time.
The following code stores the SafeArray into an array cast, then copies the array contents into a byte array:
// Results in a binary (SafeArray) response from COM object
object zipContent = wwUtils.CallMethodCom(this.HelpBuilder,
"CreateHelpZip",
this.Mode,
this.ProjectTitle, "MSDN",
dlPath + sourceFile, 50);
// *** Must convert to Array first then copy out - note 1 based
Array ct = (Array)zipContent;
Byte[] content = new byte[ct.Length];
ct.CopyTo(content, 0);
Response.ContentType = "application/x-zip-compressed";
Response.BinaryWrite(content);
Thankfully this works without any additional casts. Another option would be to loop through the array and retrieve each element and BinaryWrite() that out 1 byte at a time, which would help avoid the double memory hit of a copy. In the above code this might actually be useful because the output can be about a megs worth of data.
Ultimately the cleaner solution would be to have the COM server's code output directly to a file which can then be streamed directly by IIS with TransmitFile, but for the moment I don't want to muck with the COM server's interface.
Anyway - thanks to Christof and Kevin who pointed me in the right direction. Ah yes, this WebLog does have it's rewards at times. <s>
August 24, 2008 @ 5:46 pm
- from Maui, Hawaii
Last week I started looking into a problem that causes my IIS Worker processes to have errors on shutdown. After some mind numbing debugging I finally seem to have traced the problem down to COM object invokation and a handle leak that results because of it. What I ran into here though is very odd as it appears to be not specific to my code but a general handle leak when instantiating EXE servers.
First off let me say C++ and COM are not my forte especially now after having been away from it for a long time. But I've boiled down the following problem to its bare minimum and I'm completely stumped as to why the code would leak handles because it's such fundamental COM code.
Ok so the scenario is as follows: I have a COM client (in my 'real' app it's the ISAPI extension) that is instantiating EXE COM servers and then shortly after releasing them all happening on a single thread. This scenario occurs for certain long running processes that go off and run outside of the normal thread pool. However, the problem is reproducible without any special thread related issues interfering.
In any case the operation of the servers works perfectly fine - servers load and are properly unloaded when inst->Release() is called. The problem is that there is a Windows Handle leak.
As I was debugging my app I noticed that the code that loads EXE COM instances was effectively leaking a Windows handle when I checked in ProcessExplorer. So I kept simplifying the code more and more until I ended up with nothing more than a call to CoCreateInstance() and a inst->Release() call in COM, which in effect leaked a handle. It leaks a handle when inst->Release() is called and it does this only on EXE servers, not on DLL servers.
I managed to boil the code down in a simple C++ Console application:
#include "stdafx.h"
int _tmain(int argc, _TCHAR* argv[])
{
// COM Instance Handle Leak Bug
IUnknown *pComObj = NULL;
//IDispatch *pComObj = NULL;
CLSID clsid;
HRESULT hr = CLSIDFromProgID( L"VisualFoxPro.Application",&clsid);
CoInitialize(NULL);
char input[80];
for(int x=1; x< 10; x++)
{
// *** Commenting the next two lines out results in no handle leaks
// *** Running these two lines will leak one handle per request
hr = CoCreateInstance(clsid, NULL, CLSCTX_SERVER, IID_IUnknown, (LPVOID *) &pComObj);
//hr = CoCreateInstance(clsid, NULL, CLSCTX_SERVER, IID_IDispatch, (LPVOID *) &pComObj);
DWORD count = pComObj->Release();
printf("%d: %d\r\n",x,count);
scanf("%c",input);
}
printf("Completed.");
scanf("%c",input);
CoUninitialize();
return 0;
}
When I run this code and check the handle count in process Explorer I see the handle count increase on every single call to pComObj->Release().
It doesn't matter what EXE server I use here. I'm using a custom COM server of mine, or a more generic server above. Tried a tiny little ATL server, tried a few others like SnagIt, Word, Excel - the behavior is exactly the same - they all leak a handle.
Taking a closer look at the handles that are leaked on Release in Process Explorer (View | Lower Pane | Handles then Show unnamed handles) results in a <Unknown handle> created:
So as I said at the outside I'm not a C++ wizard, but this seems like a pretty fundamental problem given how basic this code is. Create an instance and release it - can't get any easier right? I've never noticed this before because I never debugged this all the way, but in testing I'm now seeing that the same behavior occurs on XP, Windows 2003 server and on my dev setup on Vista (as well as on Server 2008). I also tried the more round about DCOM mechanism using CoCreateInstanceEx() but the results there are the same.
I just can't make sense of what's happening here as the code is about as fundamental as it comes - make a call to CoCreateInstance() which ups the ref count, then call Release() to unload the server and all resources associated with it. The actual COM server is behaving properly, but its the damn handles that are being left behind.
What could possibly be wrong with this code? What else could I possibly do to release a handle that I don't even hold in my code? Is there some other way to release resources? It would appear that there's some issue in the DCOM proxy clean up that occurs when the server is released, but I'm not sure what that would be.
Are there any COM wonks that can point at the error in my ways?
I've attached the C++ project and EXE if anybody's adventurous - if you check this out you may have to change the server name to an EXE server installed on your machine (Word.Application or whatever that exists - although Word is problematic because it doesn't actually unload unless visibility is set off or Close() is called).
August 22, 2008 @ 2:37 pm
- from Maui, Hawaii
So I have an odd issue with an ISAPI DLL that one of my products uses. The ISAPI interface provides a gateway interface to application servers and the actual module itself works fine without any problems. The problem is that the module is apparently crashing occasionally when the Application Pool is shutting down, but it's crashing after any of the DLL code has long stopped running.
However, on IIS7 (and in some cases on IIS6 as well), the ISAPI module is causing crashes that are showing up in the event log. The error in the event log ends up like this:
The errors are rare and only occur when the Application Pool is unloaded and even then not every time. It tends to require that the app's been running for a long time before I see the errors. The Application Pool is on auto-recyle every night and that why I end up with these entries in the Event Log, but there are other situations where the Application Pool gets restarted obviously.
The above is not really causing a problem - IIS just keeps on ticking, but a number of customers have seen the event log entries and complained mainly because it's not obvious whether these are operational failures or shut down failures.
The odd thing is that this happens on shut down after all user code is long done executing. I have cleanup code in TerminateExtension() as well as some last minute clean up code in the DLL's PROCESS_DETACH handler all of which runs without any problems when I enable logging and output. I also use COM Interop in this ISAPI extension and a number of COM servers are loaded and managed in a native thread pool. The extension can also run servers in a non-COM mode, in which case the failures go away. So it appears the problem is related to the COM server references possibly not getting cleaned up properly, but since the error is occurring completely outside of my application's domain (ie. after DLL_PROCESS_DETACH in DllMain) I have no way to see what's actually leading up to this crash.
I've attached a debugger and am trying to debug this issue, but the debugger won't hit on the failure either because it appears while the process in the middle of a shutdown.
I'm also pretty baffled by the error message. Notice that it talks about wc.dll_unloaded which appears to be some system generated placeholder file rather than my actual dll:
Faulting application w3wp.exe, version 7.0.6001.18000, time stamp 0x47919413, faulting module wc.dll_unloaded, version 0.0.0.0, time stamp 0x4441d309, exception code 0xc0000005, fault offset 0x0031e6ba, process id 0x7e0, application start time 0x01c9032a50903e70.
I did a bit of searching around for this issue and it looks like my extension isn't the only one that this is happening to - a number of other ISAPI connectors (some versions of PHP for example) also seem to be suffering similar problems.
Wondering if anybody has run into this issue before and maybe dealt with this before. I'm kind of stumped on this one - I don't even have a good idea how to further debug this other than go randomly hunting for COM reference cleanup. There are some tips on how to get a stack trace to try to isolate the component that's failing but by way of the above error message the component is pretty clear <s>. Question is why this weird DLL placeholder is bombing and why it's doing this AFTER the app is practically completely shut down. Even assuming some sort of hung COM reference - those types of errors typically occur prior to DLL_PROCESS_DETACH.
Stumped.
Incidentally I have a .NET module that basically uses a very similar approach to manage the COM Servers and it works without any problems on IIS shutdown. But the .NET code is also a heck of a lot cleaner than the horrendous ISAPI code. Unfortunately the .NET module is not a 100% solution because of the dreaded tie to an IIS virtual/Appdomain <shrug>
August 21, 2008 @ 9:30 am
- from Portland, OR
In my client apps I often have a need to center elements in a browser window. Specifically when I have pop ups come up for the first time or when special status messages that are popped onto the screen during long operations, or modal dialogs pop up it's often necessary to have the content centered in the browser window. All of those operations tend to require some sort of centering of content at least in its default configuration.
I've had a centering routine for a while, but thought I should simplify by using jQuery and adding a few features like the ability to center content inside of other container elements on a page. There are already a couple of center like plugins available, but neither of the two I tried worked properly for centering content both horizontally and vertically in the main window including catching for scroll position.
One of the really nice things about jQuery is the ease with which you can extend jQuery's functionality via plug-ins. So I created a quick plug-in that does the centering job a lot more easily than my old code did and it's more flexible to boot.
Using the plug-in is as easy as:
$(document).ready(function() {
$("#box").centerInClient();
});
which centers the element named box in the client window. To center the element inside of another container element you might use:
$("#box").centerInClient({ container: window, forceAbsolute: true });
Here's the code for my centerInClient plugin:
$.fn.centerInClient = function(options) {
/// <summary>Centers the selected items in the browser window. Takes into account scroll position.
/// Ideally the selected set should only match a single element.
/// </summary>
/// <param name="fn" type="Function">Optional function called when centering is complete. Passed DOM element as parameter</param>
/// <param name="forceAbsolute" type="Boolean">if true forces the element to be removed from the document flow
/// and attached to the body element to ensure proper absolute positioning.
/// Be aware that this may cause ID hierachy for CSS styles to be affected.
/// </param>
/// <returns type="jQuery" />
var opt = { forceAbsolute: false,
container: window, // selector of element to center in
completeHandler: null
};
$.extend(opt, options);
return this.each(function(i) {
var el = $(this);
var jWin = $(opt.container);
var isWin = opt.container == window;
// force to the top of document to ENSURE that
// document absolute positioning is available
if (opt.forceAbsolute) {
if (isWin)
el.remove().appendTo("body");
else
el.remove().appendTo(jWin.get(0));
}
// have to make absolute
el.css("position", "absolute");
// height is off a bit so fudge it
var heightFudge = isWin ? 2.0 : 1.8;
var x = (isWin ? jWin.width() : jWin.outerWidth()) / 2 - el.outerWidth() / 2;
var y = (isWin ? jWin.height() : jWin.outerHeight()) / heightFudge - el.outerHeight() / 2;
el.css("left", x + jWin.scrollLeft());
el.css("top", y + jWin.scrollTop());
// if specified make callback and pass element
if (opt.completeHandler)
opt.completeHandler(this);
});
}
The plug in can center elements either in the active window taking scroll positioning into account or center content inside of another container element.
There are several options available that can be passed using the option map.
container - optional container element to center in - defaults to window
forceAbsolute - forces the element to be centered to absolute positioning at the Body level. Use this if the element is already relatively positioned to another element and so force it to 'document' absolute positioning.
completeHandler - optional handler that is called when centering is complete. Use for trapping positioning notification.
I find this a handy feature to have around especially when working with pseudo windows and popups in client scripting.
August 16, 2008 @ 6:31 pm
- from Hood River, OR
Ah here's a silly new default in SQL Server's Management Tools: When you design a table in a database and then try to make a change to a table structure that requires the table to be recreated, the management tools will not allow you to save the changes. Instead you'll be greeted by this friendly dialog:
Notice that there's no option to save the changes - it's a hard rule that is applied upon saving and you can get past this other than back out of the dialog.
My first thought here is "Crap! Now what?" and off I go searching for an option to turn this off. Eventually I find a solution after a quick search online. As it turns out it's just an annoying configuration default setting that can be easily changed, but if you're like me and you spend a while searching around the Management Tools and finding nothing initially, I ended up eventually backing out of my initial database changes and losing a bit of work in the process. It wasn't until a bit later that I found the setting to change.
Hopefully you'll find this entry before you back out of database changes - you can get out of the above dialog, make the settings change and then still go ahead and save changes to your database.
The fix is: Go to Tools | Options | Designers | Tables and Designers and uncheck the Prevent Saving Changes that require table re-creation option:
and that does the trick.
This is a pretty harsh change IMHO. While I think it's a good idea that the tools now detect table recreation changes and can notify you, I think the better option by far would have been to pop up that initial dialog with a warning message AND provide an option on the buttons to either go forward or abort. Instead this arcane switch is going to cause some pause for most people familiar with the old tool behavior. It's not like this option is easy to find - I looked in the database options before I finally found it in the global tool options.
As it is, reverting back to the 'old' behavior now doesn't let you know that a table recreate is required either, so the behavior now is the same as was with the old tools. Here Microsoft added some useful functionality and then UI fails to expose it intelligently...
August 15, 2008 @ 1:17 am
- from Hood River, OR
Here's a scenario I've run into on a few occasions: I need to be able to monitor certain CSS properties on an HTML element and know when that CSS element changes. For example, I have a some HTML element behavior plugins like a drop shadow that attaches to any HTML element, but I then need to be able to automatically keep the shadow in sync with the window if the element dragged around the window or moved via code.
Unfortunately there's no move event for HTML elements so you can't tell when it's location changes. So I've been looking around for some way to keep track of the element and a specific CSS property, but no luck. I suspect there's nothing native to do this so the only way I could think of is to use a timer and poll rather frequently for the property.
I ended up with a generic jQuery plugin that looks like this:
[updated based on comments using DOMAttrChanged and propertyChange events]
(function($){
$.fn.watch = function(prop, func, interval, id) {
/// <summary>
/// Allows you to monitor changes in a specific
/// CSS property of an element by polling the value.
/// when the value changes a function is called.
/// The function called is called in the context
/// of the selected element (ie. this)
/// </summary>
/// <param name="prop" type="String">CSS Property to watch. If not specified (null) code is called on interval</param>
/// <param name="func" type="Function">
/// Function called when the value has changed.
/// </param>
/// <param name="func" type="Function">
/// optional id that identifies this watch instance. Use if
/// if you have multiple properties you're watching.
/// </param>
/// <returns type="jQuery" />
if (!interval)
interval = 200;
if (!id)
id = "_watcher";
this.each(function() {
var _t = this;
var el = $(this);
var fnc = function() {__watcher.call(_t, id) };
var itId = null;
var z = el.get(0);
if (typeof(z.onpropertychange) == "object")
el.bind("propertychange", fnc);
else if ($.browser.mozilla)
el.bind("DOMAttrModified", fnc);
else
itId = setInterval(fnc, interval);
el.data(id, { id: itId,
prop: prop,
func: func,
val: prop ? el.css(prop) : null
});
});
return this;
}
$.fn.unwatch = function(id) {
this.each(function() {
var w = $(this).data(id);
$(this).removeData();
if (typeof (z.onpropertychange) == "object")
el.unbind("propertychange", fnc);
else if ($.browser.mozilla)
el.unbind("DOMAttrModified", fnc);
else
clearInterval(w.id);
});
return this;
}
function __watcher(id) {
var el = $(this);
var w = el.data(id);
if (w.prop) {
var newVal = el.css(w.prop);
if (w.val != newVal)
w.val = newVal;
else
return;
}
if (w.func) {
var _t = this;
w.func.call(_t)
}
}
})(jQuery);
With this I can now monitor movement by monitoring say the top CSS property of the element. The following code creates a box and uses the draggable (jquery.ui) plugin and a couple of custom plugins that center and create a shadow. Here's how I can set this up with the watcher:
$("#box")
.draggable()
.centerInClient()
.shadow()
.watch("top", function() {
$(this).shadow();
},70,"_shadow");
...
$("#box")
.unwatch("_shadow")
.shadow("remove");
This code basically sets up the window to be draggable and initially centered and then a shadow is added. The .watch() call then assigns a CSS property to monitor (top in this case) and a function to call in response. The component now sets up a setInterval call and keeps on pinging this property every time. When the top value changes the supplied function is called.
While this works and I can now drag my window around with the shadow following suit it's not perfect by a long shot. The shadow move is delayed and so drags behind the window, but using a higher timer value is not appropriate either as the UI starts getting jumpy if the timer's set with too small of an increment.
This sort of monitor can be useful for other things as well where operations are maybe not quite as time critical as a UI operation taking place.
Can anybody see a better a better way of capturing movement of an element on the page?